Exactly!
I've been fortunate enough that it's never happened to me, but I learned at a very early age to double check that the old o-ring didn't stay behind before installing a new filter. One of my first paying jobs that allowed me to do some work on cars was at a neighborhood service station. (Remember those?) One of my friends and coworkers did an oil change on a longtime customer's almost new Pontiac Grand Prix one evening. The next day the car came in on a tow truck after the customer noticed a puddle of oil in his driveway. He was smart not to start it because there was only about 1.5 quarts left in the motor.
After the boss calmed down enough, he explained what caused the leak and showed us. Fortunately there was no significant damage done.