Tired of junk thermostats. Going with NOS. Do I need to drill a hole in in?

Are you checking the stats function in a pot of boiling water?
Also its pretty easy to throw a $5 stat into an old Mopar box call it NOS and charge $$$.
As to your question, yes to hole, but 1/16 will do, I've done it.