As above, the secondaries are not any where near to being active in this phase of operation. Your carb's issues are probably in the transition phase, between idle and full primary operation. This is verified by not seeing any improvement with the primary jet changes.
I would first be working on the secondary throttle stop position and then readjusting the idle around that. The secondary stop screw is perhaps a bit to far open and sucking in more air than the primary transition slots can compensate for.
The next are is the idle air bleeds which are changeable. The point would be to enrich the idle well mixture, which is what feeds the transition slots in the early part of the transition to the primary mains operation. You can temporarily enrich the idle circuit by inserting some fine wire down through the primary idle air bleeds to block off a part of the flow area; try to block maybe 20% of the area as a test. (I wrap the wire around the air cleaner stud to keep it secure.) Then drive and see if this has changed the situation; if it does, then you know to buy and try some new idle air bleeds. If you don't have that fine wire, then buy a few steps smaller air bleed and try that. BTW, every time you change the idle air bleeds to change the transition mixture, you need to re-adjust the regular idle mixture.
I see that some of these carbs also have a 3rd set of air bleeds, for the transition circuit. If this is the case, then I would work on those rather than the idle air bleeds for this problem.
One other area is to mod the air bleeds in the emulsion tubes in the metering block. But this does not sound like where your issue lies; that area more effects going from mid load to heavy load conditions.
Having a manual trans is exacerbating the engine surge due to drivetrain slop; but the missing at mid RPMs with the clutch in shows that the engine is indeed going lean. With all the adjustments on this carb, then it is not unusual to have to change a lot of small parts once on the road and under real world loads. (And as said, keep in the back of your mind that this can be in the ignition..... the needed spark energy to fire of the mixture varies all over the map with changing loads and mixtures.)
