Mineral spirits comonly known as Varsol is petroleum based, therefore leaving the oily residue as stated above. Another alternative would be naphtha.
You can try to filter the dirty solvent through a few coffee filters into a diferent pail to remove some of the grit, then clean the sludge...