Question for those that swapped a big block into their A body.

Yes, each engine family can only use certain parts from its own family. In your case, you NEED a B block distributor from a B series engine.
350, 361, 383, 400.

RB block distributors can be used if you can find the spacer to take up the room due to the height difference.