Every gear has runout in it. This makes a "tight" section and "loose" section when run with another gear, although there should be backlash all the way around. If the
oil pump shaft is "pushing" the gear up toward the distributor, it will take out the backlash between the distributor gear and the camshaft. This will greatly exaggerate the load generated in the "tight" section of the runout, thus causing your wear.
Remember the hardness requirements between camshaft and distributor gear if running roller cam made from higher alloy steel.
Sorry if I rambled, but I design gearing for a living...