Am I an idiot for wanting to boost a high compression 318?

I missed that you're in Denver- you could get away with it at low boost.
Look at it this way: performance is down because the air is thinner, but since the air is thinner it doesn't take as much power to push through it. :) Plus, you're going downhill no matter which direction you go...

That's what I was thinking to! I am currently 2 PSI under atmospheric conditions at sea level. So if I go for 4 PSI of boost I'll really be only 2 PSI over sea level atmospheric conditions