I'd suspect a restriction in the channel leading to the manifold port. In over 30 yrs. of rebuilding carbs I haven't seen one do that but something must be happening along those lines since your sure you have them correct. It's not big deal if you don't need to use that port. But you need to find out why the ported port is supplying vacuum at idle. Like 340Dart said the throttle blades must be open too far. And that can cause it to run on like you say it's doing.
BTW: for that cam to run properly the distributor centrifugal advance needs to be modified so the engine has 18-20 degrees initial timing and 34 total. Lighter springs will let the advance to come in faster and aid in power production. Lighter springs aren't mandatory but more initial advance is. Did you do that? If not that'll cause you to have to open the throttle farther to get it to idle fast enough.
