Are number matching or original components a big deal?

You're probably going to get answers all over the map on this, but much depends on what you plan to do with the car. Do you want a fun old car to drive, or are you looking at building a show car? If the latter, yeah, you'll likely want it as original as possible. Otherwise, as long as you keep the car "in its era" you should be fine. I don't see how replacing an oil-burning 273 with a 318, 340 or 360 is a problem. The 80s-vintage aftermarket stereo doesn't look out of place, heck back in the day most of us probably put one just like it in our cars...I for one did. The factory AM radio might have been ok 50 years ago, but today not so much.

I approach owning an old car to be like owning an old house (got one of them too). As long as you don't do anything that sticks out like a sore thumb it shouldn't hurt its value. YMMV