disassembly of shifter

crazy, the outside knob is loose from the shaft, the black push button is loose from the outside knob, the whole assembly moves up and down inside the shifter shaft, I've pulled like crazy and cannot get it to separate.
the set screw is completely removed.