As you are getting into it with the arm rest off, you may be able to spray WD-40 in through the small holes to soak down the latch mechanism from inside > with the spray tube on the can. (without having to remove the interior door panel)
Also can spray the striker pin door latch area from the...