Stop in for a cup of coffee

I once called a guy in the wee hours.

He said to do it again...... still not sure what that is supposed to mean. I tried to annoy him and he threw sarcasm back at me.