Rough park or neutral to drive engagement on 904

My '65 Dart GT has the 904 Torqueflite with the 235hp 273 engine. The engagement is rough when shifting from Park to Drive, or Neutral to Drive. The lowest I can get the idle adjusted to is around 650-700rpm. I have seen some specifications that show the idle should be as low as 475-500 rpm for the V8 automatic combination. I don't think the car should "jump" like it does even with my higher idle setting. The transmission works perfectly otherwise. I don't know a lot about this transmission, so I'd appreciate any suggestions.