I ran into a similar situation after installing BK and BJ carbs on my 427 about 2 years ago. Finally found that the choke mechanism inside the housing on the front carb was allowing the fast idle cam to move slightly after even slight acceleration, thus keeping the idle too high after releasing the accelerator.
Fix: Take the housing cover off, position the choke plate completely open, and then as you re-attach the internal spring to the choke arm, preload the spring to force the choke plate to remain open longer. In other words, back off the choke so there is no way the fast idle cam can effect the linkage.
Of course you will have no choke when it is cold, but with engines like that you really don't need them anyway. Heck, they sound rough at idle all the time anyway...its part of their CHARM.
If you want to check the fast idle cam before adjustment, you can carefully watch the linkage where the cam is located when you move the linkage to open the carb. You will probably see the cam rise just enough to keep the throttle plate from completely closing when released.
Good luck, Bob