The Invisible Trillion · How Much Do We Actually Need?

A Worked Example: The Compute Cost Of Building The Software

The abstract claim of the last section — that inference is cheap, and that the expensive routing decisions are commercial rather than technical — gets concrete fast if you apply it, in compute, to the single most economically valuable thing AI does, which is write working software.

Take a large enterprise system: an Oracle-style ERP suite, an Amazon-style retail and logistics platform, the kind of system a big company runs its entire operation on. A substantial one is several million lines of code; the very largest are tens of millions. Built the traditional way, at an industry productivity of ten to fifty lines of finished, tested, reviewed code per engineer per day, a three-million-line system is five hundred to two thousand engineer-years of work, spread over the better part of a decade. This is not a hypothetical. It is roughly the history of every large enterprise software product in existence.

Now price the compute for an AI agent that does the whole loop — plans the architecture, generates the code, writes the tests, runs them, finds the failures, regenerates until they pass.

Start with tokens and convert to operations. The finished three million lines, at about ten tokens a line, is thirty million tokens of output. But the agent doesn’t write it once and stop; it drafts, tests, debugs, rewrites. Be generous and assume it generates thirty to a hundred times the final codebase getting there — every discarded draft, every test, every debugging trace. That is one to three billion tokens. Each generated token costs roughly two operations per active parameter, so for a frontier model with around 10^11 active parameters, about 2 × 10^11 operations a token. Multiply through: the entire build is on the order of 10^20 to 10^21 operations.

Sit that next to a training run. A frontier run is 10^26 to 10^27 operations. Building the whole enterprise system is therefore something like a millionth of a single training run — between a hundred thousand and ten million times less compute. On the same H100-class GPU as before, 10^21 operations is roughly a hundred to a thousand GPU-hours: a handful of GPUs for a few days, or one server rack for a week. The electricity is a megawatt-hour or two — about what a couple of houses use in a month, and almost certainly less than the laptops and the office of the engineering team that would have built it the old way burned over the decade.

Here is the same fact from the data center’s point of view, which is the view this book keeps. Colossus has 220,000 GPUs. Running for a single day, that one facility does enough inference to generate the working code for thousands of complete enterprise systems — each of which used to take hundreds of engineers a decade. One large data center, one day, thousands of systems. The compute was never the constraint.

I want to be careful here, because this is exactly the kind of number that gets misused. Those operations buy the generation and testing of the code. They do not buy the things that actually make an enterprise system work: knowing precisely what to build, wiring it into the messy real-world data and the legacy systems it has to coexist with, verifying that it does the right thing under conditions no test suite fully anticipates, and owning the result when it doesn’t.

Those are slow, they are hard, and — this is the part that matters for a book about data centers — they are human. They consume no GPUs, and they do not get faster when you add a gigawatt of them. The agent hands you the code for a megawatt-hour. The code was never the expensive part.

That is the whole point, and it closes the loop with the training numbers. The one genuinely hyperscale-scale workload — training — is a few gigawatts, globally, run intermittently. The most economically valuable use of the resulting models — producing software that used to take armies of engineers a decade — is a hundred-odd GPU-hours and a couple of megawatt-hours apiece: a rounding error against a single training run, and an imperceptible sliver of one data center.

The expensive workload is rare. The valuable workload is physically negligible. Neither shape is a demand curve that needs three hundred new hyperscale data centers a year. And it shuts the last escape hatch: if the coming flood of AI-written software is real — and it is — it arrives on almost no compute. The binding constraints on turning compute into value were never silicon, and tens of gigawatts built as though they were is capacity sized for the wrong problem.

All 19 chapters of The Invisible Trillion · Library · All courses