Of course without hearing it personally all is speculation, but I think you're on the right track with the lifter. Unless you back the rockers off when you prime, you could be holding a lifter collapsed and that won't allow
oil to enter and fill it properly. I'd pull the valve cover over the suspect area and rotate the engine to put the valves at TDC compression and prime again. It should be obvious which one it is because the pushrod will be loose.