Where is this problem coming from?

Glad you found the short. The push-on connectors are the field wires and it doesn't matter which goes where. If there's only one field wire and two terminals on the alternator, then you need a jumper wire to connect the two field terminals.

Is the "new" alternator actually new or just different? What's the story on your battery - new, old?