The AI boom is a supply-chain story

Models get the headlines, but compute, memory bandwidth and power decide what's actually possible. Here's the machinery behind the boom.

The question

Every week brings a new model that is "smarter." But intelligence on a benchmark is an output. What actually produces it? If you strip away the launch videos, what physical and economic constraints decide whether the AI boom continues?

What's happening

Capability has been climbing fast — and so has the cost of producing it. The interesting thing is not the curve. It is that the curve is being bought with three scarce inputs, and all three have hard limits.

How it actually works

Training and serving a large model is bounded by three resources, in this order:

Resource What it buys Why it's scarce
Compute (GPUs/accelerators) raw training throughput fabs, packaging, export controls
Memory bandwidth tokens/second at inference HBM supply, not transistors
Power + cooling running everything at scale grid connections, water, land
  1. Compute is a manufacturing problem. The bottleneck is not design; it is who can produce and package accelerators at volume. That makes the AI frontier partly a chip-supply frontier — geography, factories and shipping, not software.
  2. Inference is a memory problem. Once a model is trained, serving it is limited less by arithmetic and more by how fast you can move weights and activations. That is why high-bandwidth memory, not raw FLOPS, often decides the user experience.
  3. Scale is an energy problem. A datacenter is a power contract with a building around it. Grid interconnection queues and cooling are now first-order constraints, and they move on the timescale of years, not quarters.

So the boom is not an abstract "intelligence" story. It is a supply chain story: fabs → packaging → datacenters → power. Every layer has its own lead time, and the slowest layer sets the pace.

The headline metric is capability. The binding constraint is almost always something physical further down the stack.

What's unknown

  • How much of the current capability gain is algorithmic versus simply bought with more compute. The two are hard to separate, and the answer changes the forecast.
  • Whether efficiency (distillation, quantization, better architectures) can outrun the physical limits, or only delay them.
  • Open: I don't yet have good public numbers on how much of the bottleneck is fabrication versus packaging versus power. That's the piece I want to read next.

Reading

  • Semiconductor supply-chain analysis for the compute layer.
  • Datacenter power/interconnection reporting for the energy layer.