That's a classic sign of one of two things. Either the starter relay is just at the end of its life span OR the battery is getting low. When the battery voltage drops, the amperage goes up and can weld the contacts together in the relay. Obviously, that's what happened, but whether it's from the battery or relay who knows? Relays are cheap enough. I'd hawk ebay for a good NOS one or, Ray (
@halifaxhops) might have some. Cheaper than a battery.