Side to side isn't a problem. Those rockers are almost self centering. A flat lobe or collapsed lifter will be loose pulling up and down and will be noisy at idle or at all engine speeds. As Garry said sounds like a broken motor mount.
I replaced a broken trans mount in October and I did buy motor mounts to replace over the winter because i saw a bit of cracking in the rubber... I can change those and see if it works.
