Motor noise, possible solution? Cam retainer plate leaking?

I've been chasing a motor noise since I got my Duster running again a year and a half ago (no garage, discouragement, lots of reasons for the delays). Not driving it any, other than around the block hoping the motor would warm up and the nosie would quit.

The motor sounds like a diesel motor, lots and lots of rattling.

Bit of history, the car was my daily driver (with this motor) until I sold it to a friend in 2000 or so. He drove it for a time with this motor, then swapped a different motor in as it is the numbers matching motor. I got the car back the summer of 2008, and put the original motor back in at that point. The shortblock was pretty much untouched other than swapping the main bolts back on for the windage tray. Put a new timing chain on it, but cam and lifters where the same as when I first owned it. Put it together, and dropped it in the car. Until now, the motor never gave either one of us any problems.

I've put new lifters in it trying to cure the noise, dropped the pan to make sure the rods weren't hitting the windage tray and checked the rods and mains, looked at the clutch and bell to make sure nothing was hitting, pulled both exhaust manifolds to check for some kind leak. Pretty much ran out of ideas.

A buddy and I spend some time on it this weekend and at his suggestion, we pulled the fuel pump and ran it on what was left in the bowls. Noise was significantly reduced. Felt it was necessary to pull the front of the motor down so off came the timing chain cover.

Just didn't see anything that made sense, so we fired it again with no timing chain cover. It looks like the cam retainer plate is warped and oil is pouring out of the drivers side of the plate. My theory is, while the idiot light is going out, the motor isn't building enough pressure to pump the lifters up and the noise I am hearing is the lifters rattling. The only thing I can think of with the the fuel pump is that it was amplifying the noise.

Does that make any sense? Would a pretty good leak at the cam retainer plate cause problems? Or do I still have a problem some place else?

The previous onwner did pull the cam at one point in time but decided not to use the cam and lifters in the other motor, so it is possible that the retainer plate is not the original one from when I owned the car. He did make sure to keep the original lifter on their original lobes.

Thanks.