shift collar moves with steering wheel

1973 plymouth scamp 318
the shift collar moves when i turn the steering wheel bad enough that when i am driving sometimes when i make a left turn it will pop into neutral.