AI & GPU infrastructure
How AI compute actually gets built.
Notes from the field on GPU clusters, from HGX nodes to NVL72 racks: networking, power and cooling, and the economics underneath it all. Written by Tom Pechulis.
Latest
- 5 min read
HGX to NVL72: What Changes When the Rack Becomes the Server
An 8-GPU HGX node and a 72-GPU NVL72 rack look like the same idea at different sizes. They aren't. Here's what actually changes in the NVLink domain, the network, power, cooling, and how you plan a deployment.
#gpu-clusters#nvlink#nvl72#hgx
- 4 min read
Power Is the Bottleneck: A Back-of-Envelope Model for AI Data Center Capacity
How many GPUs fit in a 100 MW data center? A simple, honest model from utility power to GPU count, with the assumptions spelled out so you can swap in your own.
#power#cooling#data-centers#economics
- 4 min read
800 VDC and the 200 kW Rack: Rethinking Power Delivery for Vera Rubin
Vera Rubin NVL72 pulls up to 227 kW per rack, and Kyber is headed further. At 54 V that's more than 4,000 amps of busbar. Why the industry is moving to 800 VDC, what changes in the facility, and the economics behind it.
#power#racks#data-centers#vera-rubin#800vdc
- 4 min read
Hyperscaler Capex: The $730 Billion Question
Amazon, Alphabet, Microsoft, and Meta guided to roughly $720–745B of 2026 capex. Capex now exceeds operating cash flow at two of them. A look at the numbers, the accounting quirks, and what to watch going into Q3 prints.
#economics#capex#hyperscalers#finance#data-centers
- 4 min read
Co-Packaged Optics Goes Mainstream: The Optics Bill Gets Rewritten
NVIDIA's Spectrum-X Photonics hit full production in August. Why moving the optics onto the switch package matters for power, reliability, and the supply chain, and the CPO-vs-NPO fight coming for Rubin Ultra scale-up.
#networking#optics#cpo#supply-chain#power
- 4 min read
GPU Cloud Economics: What a GPU-Hour Really Costs
H100 rental prices went up in 2026, not down. A bottoms-up model of $/GPU-hr for a GB200 build, CoreWeave's P&L as a case study in depreciation and interest, and why $2.50 vs $3.50 is the line between a dud and a deal.
#economics#gpu-cloud#neoclouds#tco#finance
- 4 min read
Training vs Inference: Two Different Infra Problems
Same GPUs, very different clusters. Training is a synchronized throughput problem. Inference is a memory-bandwidth and KV-cache problem, which is why prefill/decode disaggregation is taking over. The math, with Llama 3 70B as the example.
#training#inference#gpu-clusters#kv-cache#economics
- 4 min read
Checkpoint Storage That Doesn't Stall Training
Meta's Llama 3 run hit an interruption every ~3 hours on 16K GPUs. At 100K GPUs, checkpoint and restart time decide your goodput. A worked model of checkpoint size, bandwidth, and optimal interval, plus what it costs when you get it wrong.
#storage#training#gpu-clusters#reliability#economics
- 4 min read
Fat Trees, Leaf-Spine, and How Many Switches You Actually Need
A worked example of sizing an 8,192-GPU scale-out fabric: rails, radix, oversubscription, switch counts, and the optics bill nobody budgets for.
#networking#gpu-clusters#fabric#planning#optics
- 3 min read
HBM Is the Real Capacity Constraint
GPU supply gets the headlines, but memory is the gating item. SK hynix just posted a 76% operating margin, Micron is booked through 2027, and Amazon raised capex because memory got expensive. What that means for how you plan clusters.
#hbm#memory#gpus#supply-chain#economics
- 4 min read
InfiniBand vs Spectrum-X: Picking a Scale-Out Fabric
Quantum-X800 InfiniBand and Spectrum-X Ethernet both ship 800 Gb/s AI fabrics. Ethernet is winning on share, InfiniBand is having a rebound, and the right answer depends on operations, tenancy, and how much you value SHARP.
#networking#infiniband#spectrum-x#gpu-clusters#ethernet
- 5 min read
Why Liquid Cooling Is No Longer Optional at AI Densities
The average data center rack still pulls about 9 kW. A GB200 NVL72 pulls 120+. Here's the physics, the facility changes, and the money behind direct-to-chip liquid cooling.
#cooling#liquid-cooling#data-centers#racks#economics