Can I remove PS pump?

Well you can, but if you're going to the trouble it might be worthwhile to ditch the power steering box and put a manual one in.