Need help removing shifter knob on 71 Duster

The black cap is probably not lining up, there might be crud keeping it from going down all the way into the handle.
Here are some more pics of the keyd notch.