The Invisible Trillion · Data Centers: The Cathedrals Nobody Visits

The Cooling

Because chips waste enormous amounts of their input energy as heat, cooling is most of what a data center does.

The basic method is air cooling. Cool air is supplied from under a raised floor or from overhead ducts, flows through the servers, picks up heat, and is exhausted into the hot aisle. From the hot aisle, it returns to a cooling unit — essentially a huge air conditioner — which removes the heat and sends cool air back.

For big data centers, the heat removed from the air has to go somewhere. This is where the environmental questions from Chapters 5 and 6 come back in. The heat can be dumped into the outside air through a cooling tower, which uses evaporating water to carry the heat away (efficient, but uses water). Or it can be dumped into a closed loop of water that’s cooled by mechanical chillers (uses less water, but uses more electricity). Or, in cold climates, it can just be blown out into the cold outside air for most of the year (“free cooling”), which uses little of either.

Newer data centers — especially ones built for AI workloads, which produce much more heat per rack than traditional workloads — increasingly use liquid cooling. Water or other coolant is piped directly to the chips, either to cold plates that sit on top of the chips or, in the most extreme version, by immersion cooling: the servers are literally submerged in a specialized fluid that conducts heat but not electricity. Immersion cooling looks exactly as science-fictiony as it sounds: racks of server parts sitting in what looks like aquariums full of mineral oil. It is becoming more common, especially for AI.

AI workloads, in general, are pushing data center design hard. A traditional rack of servers drew maybe 5 to 15 kilowatts. An AI rack can draw 80 to 100 kilowatts — an order of magnitude more, in the same physical space. The cooling systems are having to be redesigned around this, which is part of why new AI-oriented data centers look different from older general-purpose ones.

All 19 chapters of The Invisible Trillion · Library · All courses