I would say that the one place to not go cheap is on safety items for your car. You can find inexpensive rebuild parts for your brakes for example, but i wouldn't use old worn out and cracked linings and brake hoses. You have to be reasonable.
There is a big difference between good used and refurbished and unsafe.
A good example is cosmetics on your car like paint and body. While it is not good to say bondo up a rust hole, it is acceptable to cut out the affected area and weld new metal in without replacing the whole panel. Don't be afraid of a little filler to make it smooth. And as for paint, a lot of the lower line products can be just fine for a driver.
Where you run into problems overall is when you want a 100% resto that is perfect in all ways so you can win a gold award at the Mopar Nationals, and want to do it on a budget of 10 grand. But you can have a nice, safe, and good looking car that is reliable for that. Just it probably won't be a Hemi Challenger, it most likely will be a 318 Duster that had a solid body to begin with.
You must learn how to work on your car yourself, paying other people to do everything is where you spend a lot of money on a car, not really parts.
Shop around for the best deal, then be reasonable in your expectations of the outcome.
If you check out my build thread, i have put my Dart Sport on the road with a lot of good used parts that i had and some new stuff where it counted. It drives great, is reliable, and most importantly, it is safe. I have about 5 grand into it as it sits, and i did 90% of the work myself with some help from friends.
It needs paint and body and a few minor things, but i doubt that i will have over 10 grand into it when finished, just a lot of work.