Am I Being Unreasonable? Original Owner Wants Car Back

I'm not asking much above actual cash I've spent all in the last 6 months. There's no reason why I should take a loss for something I didn't initiate. If he wants to interrupt my progress, he can assume my accumulated position & do everything possible to keep me on the trajectory I've worked hard & paid actual cash to establish. If I attached a labor rate to all the work I've done, the number would be much higher, I'm positioned at just my cash outlays. Is what you're saying by working with him that I should be willing to dip into my pocket to make this work for him?
I hope the next time I buy something its from someone as straight up, honest as you

You've built this car into what you want and no reason to take a big loss when you aren't even looking to sell it
The fact that you didn't even figure labor into whatever price you gave this guy is very generous

Frankly I think there is a chance this buyer is trying to hustle you.
He wants the car for sentimental reasons, wants to pay whatever number he has in his head which he hasn't told you.
If it was me and I threw a price at him which he told you that he is unwilling to pay, there is no more reason to have further communication with him and please do not feel any remorse