Difficult choice to make without a doubt, but I think you made the right decision for your family's future. Nothing wrong with the Duster in my estimation, no, it's not a 71 Cuda, but it's something that you can enjoy with your family instead of stressing about how you can manage to buy a home and spend a lot of money on a project car at the same time.
