no you don't have to loosen to remove the axle. There is or should not be much/or any pressure on the adjuster because that is how you set "end play" (slop) in the axle bearings.
However, you may need to backoff on the adjuster to reinstall the axle and then sneak up on gaining correct end play
