'64 Dart GT: 273 for 318 poly ?

According to the research I've done, the '64 273 shares the 318 poly crank, and possibly other parts, designs, etc. from it. I'm wondering then, could the 273 in a '64 Dart GT 4-speed be swapped with with a 318 poly ('65 truck) ? Any issues with motor mounts, clearances, etc. ? Custom headers needed ? Any other snags / concerns ?