Charging voltage too high.

Would a switch interrupting the ground be ok or do I need to locate why it's feeding that 12v?
The proper approach would be to trouble-shoot to find what is wrong, rather than kludge a fix. I have a relay for underhood IGN power in my 3 1960's Mopars, indeed installed a fuse/relay box for a 1990's Jeep so relays for many things like headlamps, starter, horn, ... I removed the OE relays for those functions on the firewall to reduce clutter.