70 Dart shift column rotating???

Hello folks....I've a 70 dart which was originally column shift with lever. 10 years ago I removed the lever and installed a floor shifter. The other day I was turning the steering wheel and I actually watched to my horror as the whole back part of the housing (where the Shift lever attached long ago) rotated. I've never had the column apart past the ignition Switch..so before i go looking and take the column apart....Can someone with more experience than myself point me in the right direction??