I used to do that too, and it did work.
(spit in it, wipe it around, and a super quick rinse) not enough to wash it off.
What worked better was the antifog drops made for scuba masks.
Other than that, makes some bigger vent holes in the sides of the goggles.