Opening up 318 ports to 360 size

I have a Holley Street Dominator intake that has 318 sized ports. I was wanting to open them up to match the ports on my Speedmaster heads which are 360 sized. It looks like there's enough meat to open the ports to the sides but they need to be taller too. Problem is that the opening them up in height on the roof would leave maybe about 1/8" material. I think that's too thin.

Should I just open them up to the side and not worry about the roof or not open them up at all? A new manifold with proper sized ports is not really in the budget right now.