Just a followup. My drill press was markedly improved with removing the burr in the fit, BUT, it wasn't perfect. Recently I was just playing around and removed and re-inserted a bit numerous times. Every time it was still off center but by different amounts. Even though it looked great, it was the chuck. Just replaced the chuck and the arbor and now it drills straight.
