Sound deadener

I am ready to install some sound deadener on the doors of my Duster, and would like some advice. Is it better to apply the sound deadening material (I have the adhesive sheets) to the door, or the door panels? My door panels are homemade ones, but I think they look better than the OEM ones. I used fiberboard, cut them to the exact shape of the old panels, cut out all the appropriate holes, and covered them with black vinyl, which was backed by some cotton batting.

To me, it would be easier to apply the sound deadening material to the panels, but I'm sure that there would be a problem with that method.

Thanks!!!