Drum brake weights interfering with cragar ss wheels

Well, I have been removing them since 70's without any ill effects.