HBM Is the Real Capacity Constraint

If you only watch GPU announcements, you'd think the AI supply chain is an NVIDIA story. Look at the memory makers' July numbers and it's obvious the constraint has moved. High-bandwidth memory is the scarcest part of the accelerator right now, and it's starting to show up in hyperscaler capex lines.

TL;DR

• SK hynix 2Q26: revenue ₩79.3T (+257% YoY), operating profit ₩60.5T at a 76% margin (SK hynix). HBM4 mass shipments started in Q2.
• Micron FQ3 (May qtr): revenue $41.5B, and HBM3E and HBM4 fully booked through calendar 2027, with demand stretching into 2028. Micron now expects the HBM TAM to "easily" cross $100B in 2027, a year earlier than it previously said (Micron call).
• Amazon raised 2026 capex from ~$200B to ~$220B and specifically pointed to higher memory costs (Reuters; datacentres.com).

Capacity and bandwidth do two different jobs

On-package HBM decides two separate things:

• Capacity (GB per GPU) decides what fits without offload: weights, optimizer state, activations, and for inference, the KV cache. Long-context, high-concurrency serving is basically a KV-cache sizing exercise (a single 128K-token sequence on a 70B-class model can need ~40 GB on its own).
• Bandwidth (TB/s) decides how fast memory-bound kernels run. Decode reads every active weight for every token, so tokens/sec per GPU at low batch size tracks HBM bandwidth much more than peak FLOPS.

Vendor peaks per GPU: H100 SXM 80 GB / 3.35 TB/s (NVIDIA H100), H200 141 GB / 4.8 TB/s (NVIDIA H200), B200 192 GB / ~8 TB/s, Blackwell Ultra up to 288 GB. On Rubin, NVIDIA specs the Vera Rubin NVL72 rack at 20.7 TB of HBM4 and ~1.4 PB/s (NVIDIA Vera Rubin NVL72). Divided by 72, that's ~288 GB and ~19–20 TB/s per GPU, about a 2.4x jump in bandwidth over Blackwell for the same capacity.

That bandwidth jump is the real Rubin story. Capacity stayed flat at 288 GB, so per-GPU KV-cache headroom doesn't really grow from B300 to Rubin. What gets faster is decode.

Why HBM is so hard to make more of

The supply side is structurally slow, for three reasons:

1. HBM eats wafers. Each HBM bit uses much more DRAM wafer area than a commodity DDR5 bit. Micron's EVP of global operations said on the FQ3 call that the industry shift to HBM, with an even "higher trade ratio... expected in the future", means more silicon per bit and pushes DRAM bit costs up in the near term. Every HBM stack takes capacity away from server and PC DRAM, which is why conventional DRAM prices are spiking too.
2. Greenfield fabs take years. Micron's FY27 capex is going above the mid-$40B range, and more than half of the increase is construction. Its own commentary is that new greenfield capacity doesn't meaningfully add bits until calendar 2028.
3. Stacking and packaging yields. HBM4 means 12-high stacks, a new base die, and faster pin speeds. SK hynix says HBM4 production yields are already close to mature HBM3E levels (Yonhap), which is a big W. Even so, every vendor is selling out.

The contracts are changing too

The biggest structural shift is commercial. Micron described its new Strategic Customer Agreements as five-year, take-or-pay, non-cancelable contracts with price floors and ceilings, backed by ~$18B in customer cash deposits (about $22B including other financial commitments). SK hynix says it has signed long-term agreements with around 10 key customers.

Memory used to be the most cyclical commodity in semis. Now hyperscalers are prepaying for it like it's power or land. That tells you how the buyers rate supply risk.

What this means if you plan clusters

1. GPU delivery dates are memory delivery dates. When a vendor quotes a ship date, ask which HBM generation and stack height it assumes, and who the memory supplier is.
2. Expect memory to drive pricing. Amazon's $20B raise is the clearest public signal: the same capacity costs more because memory costs more. Budget for it.
3. Pick SKUs by GB and TB/s, not GPU count. For inference-heavy fleets, a higher-memory part can mean fewer GPUs per replica and fewer replicas overall. Model $/GB-of-HBM and $/TB/s alongside $/GPU-hr.
4. Software is the free capacity. KV-cache quantization, paged attention, prefix caching, and offloading KV to host DRAM or NVMe all stretch the HBM you already paid for. They're the cheapest "new supply" you'll find.

Bottom line

HBM went from footnote to the main event. The GPU may be the brand everyone talks about, but the stack of DRAM next to it sets how many usable accelerators ship, how much they cost, and how fast they decode. Watch the memory makers' calls as closely as NVIDIA's. Lowkey, they're the better leading indicator.