The Requirements-and-Consequences Matrix

Chapter 1.1 established the governing input — the workload archetype — and walked the cascade qualitatively. Here it becomes an engineering instrument: a requirements-and-consequences matrix that takes one archetype and returns a concrete, numbered design basis for every subsystem that follows. Where 1.1 said "pre-training implies liquid cooling," this chapter says which inlet temperature, which flow rate, which floor-loading class, which blocking ratio, which storage throughput floor, and which redundancy tier — along with the downstream cost of each cell you fill in wrong.

The altitude here is lower than in 1.1. We move through four mappings in the order an engineer actually commits them — density to the cooling cliff, fabric and the GPU:CPU/memory/storage ratios, storage and redundancy against interruption tolerance, and siting as power-first versus latency-first — and close on the reference design-basis sheets that capture all four per archetype. Read this chapter with Chapter 1.1 open: this is the table its cascade was promising.

The first irreversible commitment is the cooling modality, and it is set by one number: peak rack density. Air saturates as a heat-removal medium around ~41 kW/rack under realistic containment; rear-door heat exchangers (RDHx) and air-assisted liquid push the ceiling to ~50–100 kW without facility water at the rack; past ~100 kW the only answer is direct-to-chip liquid (DLC). A GB200 NVL72 draws ~120–132 kW — roughly ~115 kW removed by liquid and ~17 kW by residual air — which lands it firmly past the cliff. The next generation does not soften this: Vera Rubin NVL72-class racks are ~190–230 kW, and Rubin Ultra Kyber is on a ~600 kW / 800 VDC path. The density target therefore does not influence the cooling plant; it determines it.

The consequence of mis-reading the cliff is a discontinuity, not a slope — no airflow tuning closes the ~90 kW gap between air's ceiling and a 132 kW rack (Chapter 1.1). Crossing it in a retrofit costs ~$5–6M/MW — rising toward greenfield parity at full AI density — while still stranding capacity, because the slab cannot bear ~3,000 lb wet racks, the plenum was never sized for liquid distribution, and facility water was never provisioned. The map below is the lookup; the engineering lives in Chapter 5.1 (the density wall) through Chapter 5.4 (DLC).

The second mapping is the network, and it has two parts: the blocking ratio of the back-end (scale-out) fabric, set by coupling, and the system composition ratios — GPU:CPU, GPU:memory, GPU:storage — set by the archetype's host-side and data-path demands. Both are decisions where the wrong answer wastes money in opposite directions: over-build and you pay for bandwidth that never carries traffic; under-build and you starve the accelerators you spent the most on.

Coupling sets the blocking ratio. A synchronous pre-training job spends a large fraction of every step in collectives (all-reduce, all-gather, reduce-scatter), so the back-end must be 1:1 non-blocking — typically an 8-rail-optimized fat-tree. Oversubscribe it and the all-reduce stalls, dragging model FLOPs utilization down across the whole job. Loosely-coupled inference fits inside a node or a small scale-up domain, so 2:1–3:1 oversubscription is fine — at 2:1 that trims back-end cost by roughly a third on a single-source estimate, and hyperscalers oversubscribe the top interconnect layer of even their largest H100 training clusters as aggressively as ~7:1 (Meta ran that ratio between the fat-bandwidth islands of its 24k–32k-H100 clusters), so an inference fabric that never runs synchronous collectives has ample room to loosen. Sizing a non-blocking fabric for an inference business is the cleanest example of a self-inflicted anti-pattern — bisection bandwidth the requests never use. → Chapter 8.5 (topology & oversubscription), Chapter 8.4 (protocols).

The composition ratios are archetype-specific and shifting. Training historically ran ~4–8 GPU:1 CPU; agentic inference — with host-side sandbox execution, retrieval, tool calls, and RL rollouts — is pulling that toward ~2:1 and below (Intel reports 8:1 → 1:1), which changes the host BOM and the node power budget. GPU:memory is set by per-GPU HBM (H100 80 GB → B200 180 GB → B300 288 GB → Rubin Ultra ~1 TB) plus host RAM, and inference is increasingly KV-cache-bound rather than weight-bound. GPU:storage is a bandwidth ratio, not a capacity one: it is fixed by checkpoint write speed for training and data-loader read speed for both — which is exactly where Mapping 3 begins.

Storage and redundancy are two consequences of the same input — the archetype's tolerance for an interrupted GPU — and they are most defensible when designed together. The question storage answers is: when does a GPU stall waiting on data, and what does that stall cost? The question redundancy answers is: when a node or a power feed fails, does the workload restart cheaply or lose money?

Storage is sized by the throughput that keeps GPUs fed, not by capacity alone. For training, the two binding flows are checkpoint write bandwidth — because a synchronous job pauses all GPUs to write a checkpoint, and slow writes are pure goodput loss — and data-loader read bandwidth, because a starved loader idles the whole pipeline. A high-bandwidth parallel file system feeding GPUDirect Storage (CPU-bypass) is the training default; this is the link that turns a storage decision into a GPU-efficiency decision (Chapter 9.1, Chapter 9.3, Chapter 9.4). For online inference, the new pressure is the KV-cache: reasoning models emit long decode sequences, inflating per-request cache, so the hierarchy now tiers KV state across HBM, host memory, and NVMe/CXL (Chapter 9.7). Batch and edge are the relaxed cases — throughput reads with no low-latency floor.

Redundancy is set by interruption tolerance, and over-building it is a recognizable waste. A synchronous training job already restarts from a checkpoint when any node fails — at best-in-class operators, MTBF is ~7 days per 512 GPUs, and Meta's Llama 3 405B run logged ~one interruption every three hours on 16,384 H100s — so the rational posture is N or N+1 plus disciplined checkpointing, not 2N (Chapter 1.1's over-provisioned-redundancy anti-pattern — that capital returns more as goodput). An always-on inference business inverts this: an outage is lost revenue and a breached SLA, so 2N / Tier-IV-class power with N+1 cooling on standby is justified. → Chapter 12.1 (redundancy topologies), Chapter 12.2 (goodput vs availability), Chapter 12.4 (goodput SLAs).

The fourth mapping is the least reversible of all — you cannot move a slab — which is why it must be derived from the workload, never chosen first and rationalized after. Latency sensitivity is the discriminator. Pre-training and batch inference are indifferent to user proximity, so they are scored power-first: chase the cheapest firm (or curtailable) megawatts and the coldest free-cooling climate, accept that the site may be hours from any metro, and treat the grid-interconnection queue slot as the scarcest asset in the project. Online and edge inference are scored latency-first: chase sub-50 ms reach to users and accept power that can cost 2–4x more, distributing capacity for proximity rather than concentrating it for cost.

The 2026 context sharpens this fork. The binding constraint is power, not chips — US large-load interconnection waits run ~3–7+ years in the densest hubs — so a power-first archetype that mis-sites near expensive, constrained metro power burns both money and a queue slot it cannot recover. A latency-first archetype sited in a cheap-power exurb, conversely, may meet its energy budget and miss its SLO, which is the more expensive miss because it loses the revenue the building exists to earn. Water availability is a hard siting gate for any liquid-cooled hall regardless of archetype (Chapter 3.7). The reordered hierarchy and the speed-to-power race are engineered in Chapter 3.1 and Chapter 3.2; the fiber/latency screen in Chapter 3.6.

The four mappings converge into a single artifact: a reference design-basis sheet per archetype that freezes the inherited assumptions before any long-lead equipment is ordered. This is the deliverable 1.1 promised under "design-basis document," now filled in. Each sheet pins one row per subsystem — density tier, cooling modality, fabric blocking ratio, system ratios, storage tier and throughput floor, redundancy topology, voltage class, and siting class — plus a reversible-vs-irreversible register recording which assumptions are hedged and which are committed. The table below is the skeleton; a real sheet attaches the numbers (the ramp curve, the MVA sizing, the CDU capacity) and the signatures.