I usually am in bed by 11 pm, wake up between 4-5 am to use the bathroom and then back to bed. I get up between 7-8 am to start the day.
On rare nights, it can take an hour to get back to sleep after the 4 am bathroom visit...but not often.
I have learned to not wake too much in the middle of the night (no phone use or bright lights) and try to start vividly recalling the last dreams I had before I got up as a way to expedite getting back to sleep. It seems to work for me fairly well.