Power brakes acting silly!

The vacuum booster is actually a power assist. It is supposed to provide effortless pedal operation. So lets assume with the engine off and residual vacuum used up, you pump the pedal and it sits right at the top and feels fine. Hold it there with medium pressure and wait, feel for a slow give away, pedal sinking. That would be bypassing within the master cylinder. Power assist would help you shove you foot right on through that bypass a lot faster. This is the condition I had in my Camry recently.
If the pedal doesn't move at all, start the engine. the pedal should drop a little, inch or so. That's normal. If it goes to the floor, either the check valve in the vacuum line, or the line connections, or the booster itself is failing.