LT,
what year is your CJ?
From ford big block engines; "If you plan to use any hydralic lifter engine, the lifter galleries must be plugged. Easiest way is to plug short galleries tapping off center gallery. Remove cup plugs. Drill and tap passage to accept two Allen head plugs,one obove center
oil gallery and one below. use small amount of sealer, repeat on other side."
you can pull the cam on an ERA. while in the car.
Teds solution sounds like a good one . If you have tapped and plugged galleries, the holes below the main gallery will also probably be tapped , in which case you will only need to remove the top plugs and add an allen head plug in the lower hole (again make shure it is below the bottom of the main gallery!