is there really no replacement for displacement?...

I agree with a couple other posters on here. Depends on where you want to go with your car. If you are an avid restorer and want the numbers matching and everything as original as possible- leave as is. If you are not so concerned with that, you could spend the buca bucks on a 340 $$$$

I was in the same position. I went with a 360 over a 340 because there are a lot around and you can get them for cheap.

Go with your gut!