Get a large container of Evaporust. Soak rags or small towels and line the inside of the bumper with them. Press them in firm so they find the contour.
Leave them over night. Remove rags and clean it out. Depending on the amount of rust, you may wanna do 2 or 3 applications.
I've always been amazed at how well Evaporust works. It's not a rust cure, but it will help you remove a bunch.