Do I have reason to be concerned???

It was glowing at some point based on those heat marks. Which means it got hot enough to change the hardness. Whether it actually changed the hardness or not is anyone's guess, and will depend on how it cooled. But it's a very real possibility that axle is no longer as strong as it's supposed to be. Whether or not that's a problem for you depends on the application. And whether or not the axle is still straight.