how do i remove these drums?

There's probably rust around the studs as well.
You could wack those back a quarter or half an inch, this would free up the drum's movement around the hub too.
Use a block of hardwood between the studs and hammer if you want to save them.