Any idea what causes this?

Have you replaced your PCV valve. If they are stuck open will make an internal vacuum leak. And if you have power brakes could be a leaking booster