Trying to change turn signal switch

I never have. There should be a notch or to in the bearing housing you can stick the screwdriver in a pry it up some. That will get you a little gap you can get the screwdriver in and pry it the rest of the way off.