Rocker Arm help

Is there a way to account for the thickness of the sandpaper, or does it not matter?
It matters, just have to remove mostly from the bottoms of the saddles using narrow strips so no material is removed from the sides at first.