New Pitman Arm Help Please

If the shaft was large and the arm was small wouldn't that cause the issue OP described?

It’s a 0.11” difference. little less than 1/8”

It’s not that close