will a bigger cam kill the torque?

Since you are running a 440 and IF you are at 10:1, you can get by with a bit bigger cam. Running the small cam IMO is leaving some power on the table.