Anyone ever experience this

You want a slight drag on the drum brakes. The rear drums shouldn't just pull right off with no force but you shouldn't feel like the shoes are gripping the hell out of the drums when pulling them off either. You should be able to feel that the drum and shoes are making contact when pulling the drums off and on but not enough to where a bunch of force needs to be used.

Before you attempt to pull the drums off, hit them with a mallet a few times just in case there is any rust making them feel like they're set too tight when in fact it's just the rust holding the drum to the axle. After that you should be able to slip the drum off and on with a fairly low amount of force. If it feels like the shoes are holding on to the drums for dear life, you either have them set too tight or the e brake is stuck.