Another lifter noise question

I've been fighting a lifter noise since motor was rebuilt,has had 2 different cam and lift sets in it.After sitting awhile would tap loud for only 2 or 3 secs then quiet down but still tap quietly.Lifter bores where checked when block was done.I recently installed some 273 rockers and it quieted it down at start up but still taps at idle and seems to go away under load.I did notice that on a couple of the lifters that when I went to adjust preload I could easily push the pushrod in with light pressure from the tip of my finger.Even when they're bled down should I be able to compress the lifter that easy? Checked how the cam bearing oil holes where lined up when I changed the cam,looked good.plenty of oil in the valve covers.No shavings in oil. I assume that if I wasn't getting enough oil up there they'd all be making noise.Could running the engine with too little preload mess up the lifter? Already checked/changed header gaskets.