Need help removing shifter knob on 71 Duster

Yes, I pushing and hold the button down while turning. It turn freely, so it feels like it should come off.