Grey sludge in brand new radiator - what does it mean.

"Boiling over" CAN be a sign of combustion leakage either gasket, or cracked head/ block/ etc

Look for wet / coolant on any of the plugs

Do a compression test

"Rig" your air hose to your compression tester/ cylinder adapter so you can inject air, listen and look for exit in the radiator

After that I would drain and thoroughly flush it, maybe two, three times

Obviously, also look for coolant in the oil

Last, call around to rad shops, etc to find someone with the old chemical "block check" This is a tool which sucks a little coolant in, and reacts with a chemical which turns color in the presence of combustion products