It does sound like a vacuum hose is off, or some other major vacuum leak as brian6pac mentioned.
Try disconnecting everything that has vacuum and plugging them off.
Brake booster, PCV and any vacuum hoses.
What you describe sounds like it would be a pretty big leak like a brake booster or PCV hose.