Manual steering column moves side to side?

are you referring to the bearing thats under the snap ring? I meant is it normal for the shaft to move with the turn signal housing and bearing off

Yes, because there is no longer support there with the bearing off.