The milk adds heat capacity to the beverage.

Think about it this way: say the milk is out of the fridge, its cooler than room temp, the coffee is hotter than room temp.

If you add the milk to the hot coffee immediately much of the coffees heat warms the milk. If you add the milk later instead it dissipates into the room and when you add the milk your beverage is cooler.

The same is still true for room tempered milk, just not as obviously. It still adds heat capacity. But adding it after much of the heat is gone is like adding SSDs to your server after your company already lost 80% of the customer data, its just not gonna fly.

So yeah, adding milk early gives you a warmer beverage. Adding milk later produces a cooler one.