he can't hold the earnest deposit, but if he does he also can't resell the house, as you two have a contract.
I recommend your agent tell his agent if he resells without a release, you can sue him for damages and harm, thus putting a cloud on his title. no title company will process it if they see a legal proceeding on the property.
It takes two people to initiate a contract, and two people to end it.