The bushing should be reamed to size after installation. There is a clearance spec.
That shaft should slide in easy with assembly lube. Don't force it in at all! You should be able to spin it and have it pop up with a screwdriver off the cam gear after installed to set its location.
Reaming that bushing should be done before the engine is assembled.
You do know where those bronze shavings are going when you ream it? they are going into the motor If you know someone with a lathe you could polish the shaft to a smaller Diameter. Just a thought, Or ream it and wash it down with a good flow of diesel with the oil pan plug out