Not so much that you are wearing things out as much as everything working harder.
Oil is thicker so does not flow as well or evap contaminates and moisture off. The hyd. lifters want to pump up at high revs a little easier . A cold engine is missing any safety net that an up to temp engine might have.
PS>>>> Boat engines that are run cold do wear bores and rings etc.... at a higher rate than those will a good working cooling systems. I saw this working in boat yards.