Rethinking my old 318...

I'm 29 and in college so I don't have piles of money to build engines. So here's what I am dealing with. I have a '71 Twister Duster that was my high school car circa '98/99. Me and a friend rebuilt the 318 in 2000 and now it probably has less than 2000 miles on it.
*318 stock bore and stock pistons with moly rings
*All bearings were replaced with Clevite's
*Mopar Performance Double roller timing chain
*Small tube headers
*Crane cam in the 470 lift area with 224 or so duration (Crane Fireball from way back in the day-Erson lifters)
*Melling stock volume oil pump
*'74 318 heads (they are restrictive I know but it was all I had at the time)
*Offy intake and 600 Edelbrock carb (intake performed satisfactory even though people hate it)
*Points distributor with Mallory electronic conversion
*Clutch fan.
*All else pretty much stockish. Rear is an 8 3/4 with 3.23 gears and a 4 speed in front.

Would you take and drop some milled 360 heads on the 318? Ditch the 360 heads for swirl port 318 heads???

Or just drop a 360 in it at a later date? I have one but it needs EVERYTHING.
I have some 400's to but a big block swap can't be that cheap to do from my research.

I have another 318 that is modded with 360 heads along with a 268* Mopar cam. It has other this and that mods that make it plenty strong for my '41 Dodge pickup. That one stays. Happy with it...

Just asking opinions even though most will say "go 360!!" :D