You don't need the button with green bearings and could damage the bearings if they are tight. But if you leave the button in put one axle in with the backing plate and tighten it. Then install the other without the backing plate .
Make sure the bearing retainer on the second axle fits tight against the the housing with no backing plate. This way when you reinstall the second axle with the backing plate you know you have at least that thickness of clearance for the green bearings.
Just an easy way to be sure. Better to have a little play then none at all.