Pulleys overlapping?

So, what do you suggest I do to align the two pulleys? Something to do with balancer?

Nope.
If that were mine and I wanted a simple solution I would space the water pump pulley out as far as I could without it hitting the pulley in front of it.

I'd bet your alternator pulley would be real close to good after.
A crank to water pump pulley misalignment of 1/4 is a whole lot worse that a alternator being 1/4 off because of how far apart they are.
It gives the belt a chance to correct for small variances over that distance.