Crane Rocker arm clearance on Iron heads issue

To be clear the shafts were on a set of X heads but now they are on a set 974 360 heads. What puzzles me is that the same shaft that will rub on the number 1 exhaust will not do this if I swap it to the other head. I have checked all the thicknesses on the spacers and rockers and they are all within .001 of each other. I also dont want to kick that one to the right as the roller sit of to the side on that valve.