I've been looking up a video on assembly of a column like yours and I agree. You need a drift pin flat faced punch a little smaller than the pin and hammer to drive it out.
There is a really good video of a guy with a duster with a beige interior that's excellent. Just look up "replacing ignition locking cylinder on Plymouth duster" on youtube and it should pop up. He does the whole thing, but doesn't actually show himself whacking at the pin.
I'll be giving it another go tomorrow. In the meantime, it's been hit with some PB Blaster and Marvel Mystery Oil.
