Grinding, " diesel " noise under valve covers

I was just checking a head with rockers on the bench. You can move the rockers side to side an 1/8 inch with no push rods and pressure on them. They may be no space between the rockers but they are just floating on the shaft and against each other but there's no pressure there pushing them together, when running they are coated with oil.
I can move one or them laterally, very minutely, sometimes, but.not all of em.