Manual steering column moves side to side?

Yes, because there is no longer support there with the bearing off.

lol you had me worried for a second, how much play is allowable with that bearing installed? Its enough to see but not enough to measure, also does that bearing need greased?