I quit the restoration shop.

Sounds like a case of covering his *** while hanging your *** out in the wind.
He's been saving a small fortune in Social Security taxes for you and screwing you in the process. You may think he's a great guy now, but when you have this huge hole in your work years when it comes time to retire he'll be a S.O.B.
I wouldn't hesitate to ask for unemployment, if he gets fined, then he should have been doing things legal. The kicker is, he's known since 2010 and kept working you illegally for the past 4 years. It's one thing to not know, but to continue after the fact, he needs to be busted.