How about a leak in the vacuum booster rubber line,or a bad booster check valve?. If the booster makes pressure,the check valve should release pressure unplugged. If it doesn't, then I would look at all the cheap small stuff first.
I have had cars that did that before and you are right about something in the vacuum valve causing it, but I don't remember what.
The last one, everthing would be fine till I touched the pedal and then it would suck the pedal down to the floor and totally lock the brakes.
I had to shut it off, jump out and pull the vac line off to go again, and if I touched the pedal it would do it again.
The problem was that I was not willing to risk a sudden and complete lockup of the brakes at any random speed, with who knows what as far as obstacles.
(My luck it would be at 90 miles an hour in four lanes of traffic.) :banghead:
I just replaced them.