I did this to a 74 charger back in 07/08. It's pretty straight forward, a good shop can do it and balance everything for you. They essentially just grind the main journals to the smaller B series sizes. Now a days though, the prices on 440sources stroker kits make it more economical to just get one of their kits.
As for clearances, it depends on the block, the later years(76-78) have thinner walls than other years but not by much. Mine didn't require any clearancing to be done but consult with your engine builder.
As for downsides, not many at all. A 451 (400 with a 440 crank) is gonna make good reliable and useable power especially torque.
