> I guess an issue could be heat exchange from a vacuum to whatever the cooling system is, but that's above my brain grade.

That's sort of the whole problem though. Whatever particles are around in a near-vacuum might be very cold, but there are also very very few of them. There just isn't anything to exchange the heat to. A vacuum is an insulator, that's how Thermos bottles work for example.