That is common.
The headers release too much heat, the starter gets hot, and needs more amperage from the battery to get it to turn.
If the battery is old, and you plan on replacing it, get the biggest battery that fits, the one with the highest cold cranking amps.
Also, make sure the...