Rust removal, how far????

Ok I am having trouble finding a good chemical rust remover and I have tried grinding a few places. I am using a small angle grinder with Rol-lock wheels and 36 grit. It came really clean just a few small spots smaller then a pin head. I also used one of those black stripping wheels in another spot and it left a little more. But my question is does evey little bit have to come off or is this small amount acceptable. First pics stripper wheel and second pic the grinder is one the right stripper wheel on left. You can't even see the rust in the grinder spot but they are there just really small.

Rusty inner question pics 007.jpg

Rusty inner question pics 005.jpg