which valves have cracked retainers?
Again, maybe an optical illusion, but in the picture of the rocker tip and cracked retainer, contact looks to be more on pushrod side of valve and the valve apears to be on the seat. If it is at peak lift, it would probably be even more toward pushrod side.
As for metal fatigue, I would think it is unlikely, because of the low mileage. I also think it would break in a different way.
I'm a little short on experience running a big block at the track, so forgive my ignorance, but maybe it would be a good idea to not run the springs on the rocker shaft, and maybe opt for solid ring spacers like a small block.
What rpm's are you running at the track. And what are the clearances between retainer and guide/stem seal @ full lift? what is the spring coil clearance @ full lift?
Have you taken a look in the cylinders with a bore scope on the cylinders with the cracked retainers? Maybe the valves contacted the pistons?