Help it is grinding bad

This is a guess without seeing it with my own eyes, but it sounds as if the axles were made for narrower drums. I believe the standard dimension from the bearing flange to wheel mounting flange is 2 1/2". If narrower you would need narrower brakes.