Do I have reason to be concerned???

Never in my life have I used a torch to do that. I've pressed off a crap ton of them too.

In the old service manuals they show grinding them down almost to the axial and then taking a chisel and popping them with a hammer to crack them then they slide right off.