The Air Runs Out First

The first thing nobody tells you about running your own hardware is that you will become interested in air. I put four graphics cards in a spare room, and within a week I was thinking about door gaps and return paths and where the warm air actually goes after it leaves the back of the rack. The answer, it turns out, is that it goes into the room, and then into the rest of the house, and then it is your problem in a way that a cloud invoice never is. On a long run that room hit 105 °F before I did anything about it, and when I wrote the build up afterward the honest line was that cooling had been a larger afterthought than the electrical work, and the electrical work was already the part I had not thought about.

All of which is a domestic version of a decision industrial operators make constantly, and I want to be careful about the size of the gap before I go any further. My problem is four cards and a window. Theirs is a building. But the shape is identical, and having felt the small one has changed how I read the large one: every watt that goes into a machine comes back out as heat, immediately, with no exceptions, and somebody has to move it.

The racks got much hotter, quite suddenly. A conventional server rack drew something in the range of five to ten kilowatts for years, and rooms were designed around that. A rack built for AI training draws a multiple of it, and the multiple is closer to ten than to two. The building did not change. The floor did not change. What sat on the floor started producing an order of magnitude more heat in the same footprint.

Air stopped being able to do the job. You can move a lot of heat with air if you are careful about where it goes, and the industry spent twenty years getting careful: hot aisles, cold aisles, containment, blanking panels, raised floors. Past a certain density none of that is enough, because there is a limit to how much air you can push through a rack before the fans themselves become the problem. That limit arrived faster than the buildings could be replaced.

So liquid came back, and it came back as plumbing. Water carries heat far better than air does, which is why the answer to dense racks is a cold plate sitting directly on the chip and a loop running out of the building. ASHRAE added a whole equipment class for it in 2021, with a tighter temperature band than air-cooled kit gets. The consequence is that a modern AI hall is a plumbing project with servers in it, and plumbing has failure modes that a room full of fans does not.

This is the part I did not expect and find genuinely interesting. Accelerator supply, high bandwidth memory and grid interconnection are all real constraints, and they are the ones that get counted. Underneath them sits a set of trades software people never think about: mechanical technicians, controls specialists, commissioning engineers, the people who balance a hydronic loop and know why a pump is cavitating. Those are not jobs you can spin up in a quarter, and the training pipeline for them was built for an era that needed far fewer of them.

They are also jobs with a long apprenticeship and a lot of tacit knowledge, which is exactly the kind of work that does not compress. You can teach someone the theory of a refrigeration cycle in a week. Teaching them to walk into a plant room, listen, and say "that is not right" takes years and cannot be done from a manual. The industry has spent two decades treating that skill as a facilities cost line, and is now discovering it was a capability.

There is an irony in it that I do not think is cheap. An enormous amount of money is being spent to automate knowledge work, and the physical precondition for spending it is a category of skilled manual work that nobody automated, nobody glamorized, and nobody trained enough people to do. The models cannot be run without the trades. The trades were not consulted about the models.

The other thing density changes is what a bad night looks like. When cooling fails in a room of ordinary servers you have some minutes: the air holds a little heat, the equipment tolerates a rise, and somebody can drive in. At AI density that margin mostly goes away: the same room holds a fraction of the time it used to, and the equipment is less tolerant rather than more. The failure is not gradual. It goes from fine to thermal shutdown fast enough that the response has to be automatic, because no human is getting there in time.

Which means the job on the other end of that call has changed shape. It used to be respond and fix. It is increasingly arrive and assess, because whatever was going to happen has already happened, and the person driving in at two in the morning is walking into a building that has already made its decision. That is a worse job than it was, and it is being done by more people, in more buildings, in more places, than at any point in the history of this industry.

The local objections are usually noise and water, and both have a factual answer on each side. Heat rejection equipment runs continuously and is tuned for capacity rather than quiet, which is why noise turns up in objections more than the industry expects. Evaporative designs consume water in a specific watershed; closed-loop and air-cooled designs use little or none and are increasingly what gets built, at the cost of more electricity for the same heat. Neither the objection nor the rebuttal is dishonest. They are describing different designs.

The local economics also cut both ways. Construction employs a lot of people for a while and the permanent headcount is small for the footprint, which is the asymmetry objectors point at. The property tax base and the grid investment are real and durable, which is what supporters point at, and in a small jurisdiction the tax line can be transformative. Both of those things are true at once, and which one matters more is a question about a particular county rather than about datacenters.

Compute sits inside somebody's jurisdiction, always. A server in another country answers to that country's law, including its rules on lawful access and what a court there can compel. For a great deal of regulated work - health records, defense, anything with a residency clause in the contract - that is not a preference, it is a hard stop. The country that hosts the building holds a lever over what happens inside it, and no amount of encryption changes whose courts are nearby.

In a shortage you are a foreign customer in somebody else's queue. Capacity gets allocated, and when it is scarce the allocation is political as often as it is commercial. A host government that decides its own industries, hospitals or agencies come first is not doing anything unusual; that is what governments do under pressure. Renting rather than building is a fine arrangement right up until the moment the thing you are renting becomes scarce, which is exactly the moment you need it.

And the skill only grows where the buildings are. This is the part that connects to everything above. The technicians, the controls specialists, the commissioning engineers: that capability accumulates in the places doing the work, over years, in people. A country that decides not to build this infrastructure is not only importing compute, it is declining to develop the workforce that knows how to run it - and that is the piece you cannot buy back quickly when you change your mind.

Now the part that makes my own position smaller. I run models locally, I have written about buying the hardware to do it, and every cloud service I use sits in one of these buildings. The heat I find interesting in my spare room is the same heat, and I am on the demand side of it. A person who benefits from a supply chain does not get to be a neutral observer of its externalities.

The industry also deserves more credit than a piece like this usually gives it. The efficiency work has been genuine and hard: containment, better airflow, higher operating temperatures, waste heat sold to district networks in a few places rather than thrown away. Uptime's survey has average efficiency close to flat for six years running, which reads like failure until you notice what it is holding steady against, which is a load that has grown enormously and got denser while it did.

The sovereignty argument needs its own limit, because it is the easiest one here to overstate. Most hosting abroad is in allied countries under treaties that work, plenty of workloads have no residency requirement at all, and "build it here" has been used to justify a great deal of expensive infrastructure that was never really about resilience. The risk is real and it is not a reason to accept every proposal that invokes it.

The numbers are not small any more. The next time you watch a model produce something in a couple of seconds, it is worth remembering that the same couple of seconds happened in a building, that the heat left through a pipe or a fan, and that a person you will never meet is responsible for making sure it kept doing so. The air runs out first. It always did. We just used to have enough of it that nobody had to think about the person whose job it was to move it.