Grinding when downshifting

I rarely downshift. I just let off the gas and let the engine do most of the the braking. Every time you cycle the clutch is one more time closer to replacing it
plus it puts undue stress on the rest of the drivetrain. Brake shoes are cheaper to replace than clutches and transmissions. (don't shoot me, just my personal opinion).