In car cam swap

I know there is a post about this somewhere but was unable to find it. My manifold swap post got me thinking (at the suggestion of a few others too) that just putting a 4bbl manifold and carb on a car with a stock 318 2bbl .380 lift cam is not going to do much, if anything for performance and that a cam install would need to be done in order to reap the rewards of more air/fuel.

So - anyone done this? Easy? Pain in the ***? Seems like a lot of stuff has to come off/out for this to happen. I may need to replace my water pump at some point soon since it looks like it's weeping a little, so I might as well kill two birds with one stone.

My thought though is that it's a huge job that's better suited to pulling the engine which I didn't want to do. Then there's the extra cost of all this beyond the manifold - cam and lifters, gaskets etc. which I was also trying to avoid.

Almost seems like I may be shelving this whole manifold swap idea due to runaway cost.

Thoughts? Criticism? Am I an idiot? Well, we all know that to be true but...