how bad is this and how much performance am i losing?

I agree; if you have lifter preload over .020, and it's not eating lobes or guides, then let 'er rip.