Believe it or not the factory had a large stepped reamer to fix this exact problem on assembly. The cam bores are not perfectly in line in all engines. This is why the service manual specifically says not to replace the cam bearings if they are NOT damaged. The factory line reamed them. A high end shop will have a cam bearing reamer. If the cam is straight this is the problem. So either use a bearing knife, a slotted old cam or find the "big old bearing reamer":
View attachment 1715195163
