intake manifold heater hose fitting

I'm installing a new intake manifold and the diameter of the hole for the heater hose return is smaller than the one in my old Perfomer intake.
Can I just bush up the fitting or will it hurt anything by having a smaller diameter hole?