Are number matching or original components a big deal?

This is all up to you. If it was a Hemi car I would try to persuade you to keep it factory but in this case it is up to you. If it was up to me I would build the car how I wanted and have fun with it. If you want to do a 1000 point restoration then yes that will hurt the car but if you want a good driver I would not worry about it.