paint on the inside of heads flaking

i took off my valve covers today and noticed that the paint on the heads are flaking, just by rubbing my fingers over it some paint comes off , so my main question is how damaging will this be to the motor and should i try and remove all the loose paint, why would someone paint the heads anyways (ya the outside i understand but why paint the inside were if it flakes the paint will going into the motor) the oil should prevent any rust .....