I believe it will come out. Roll pin punch set. Wrong type of punch and it will be difficult. Support handle on a block or on a vise. I usually open vise 1/4” or lessso pin will drop through jaws.
Soak the pin first. Usually these plastic handles are bent on the long end from 50 years of being pulled. They are available from aftermarket suppliers.