Yup. The A/C bracketry for the Slant-6 was not very well thought out, and this what you describe is pretty common. Yes, there were some setups in the '60s B- and C-body cars that had an idler pulley exactly where you have in mind, on its own little bracket. It worked okeh as long as the bearing in the idler held up; they got very noisy after awhile and weren't designed to be serviced. There was also an idler in that position on some of the later '70s installations, but those don't retrofit to the earlier compressor bracket, and while the later compressor bracket will bolt right onto the early engine and compressor, but the later bracket spaces the compressor further away from the engine, which on a pre-'67 A-body puts the compressor smack in the middle of the battery. So one question here is what year car you're working on.
Another thing that can help a lot with this problem is to swap on an inertia-ring style compressor clutch. These take out a whole lot of the "paint shaker" noise and vibration due to belt flap even if you don't have an idler. I think I still have one or two of those clutches on the shelf; send me a PM if you like.
