76 caliper rub on bracket

I believe I should be able to clearance the area of contact since it is actually hard to notice that it is in fact touching, it looks like it is contacting all three points at once and maybe not even touching but sliding back and forth a few times does leave marks on the center of the caliper bore exterior. Before I go grinding on new calipers, I wanted to ask to make sure I'm not missing something. I will get some pics as soon as get home today. Was a bit dark last night to get photos.