The star adjuster should only turn in one direction as the self adjusting bar will prevent it from turning the opposite way. If you can turn it both ways with the brake spoon, something is amiss in there and should be checked out.
I always have had bad luck with a brake spoon but I don't have much patience either. I always like to adjust mine while removing the drums off and on to get that perfect shoe to drum fit. You more than likely know the basics but make sure you have the adjusters on the correct side... they are labeled left and right. Something sounds fishy though so I would check them out good.