Thermal Fundamentals & the Density Wall

Every accelerator is, thermodynamically, a space heater that happens to do arithmetic. A 1,200 W GPU converts essentially all of its electrical input into heat, and that heat must be removed continuously and within a few degrees of a fixed temperature limit or the silicon throttles, ages, or fails. This is the one constraint in the building that does not negotiate. You can oversubscribe a fabric, defer a redundancy tier, re-price a power contract — but you cannot argue with the second law. The heat leaves through the path you built for it, at the rate physics allows, or the machine slows down to match the path you actually have.

Part 5 builds on this thermal foundation. It starts with heat-flux first principles and the thermal-resistance stack-up from junction to coolant, then follows the 2020–2027 rack roadmap while testing conventional air, close-coupled air, RDHx/AALC, hybrid, and DLC against named equipment and facility envelopes. The thermal metrics—approach temperature, NTU/effectiveness, airflow and pressure, coolant delta-T, flow, and pressure drop—become the qualification vocabulary. Rack kW narrows the feasible set; heat split and flux, airflow/inlet limits, FWS/TCS or entering-water conditions, room rejection, climate, service/redundancy, and the refresh tail select the service.

The governing quantity is not power but heat flux — power per unit area, W/cm². A 700 W H100 SXM module, with its full rated power hypothetically spread over roughly 8 cm², gives 85–90 W/cm² by division — a guide scaling calculation from NVIDIA’s H100 rating, not a measured die heat flux. The module’s power includes heat outside the die, and a Blackwell-class package past 1 kW still needs its own cooled area and spatial heat map before you call an average a hotspot. Flux is what the cooling solution actually fights, because heat removal is fundamentally limited by how much surface area you can couple to a coolant and how steep a temperature gradient you can sustain across it.

The chip vendor hands you two numbers that anchor the budget — the selected mode’s temperature limit and heat load — inside a wider thermal and mechanical envelope that also fixes supported interfaces and heat distribution. Tjunction-max — the maximum allowable on-die temperature, specified for the selected component and operating mode — bounds the qualified operating envelope; the component’s protection logic determines throttling or shutdown, and operation beyond its absolute limits risks damage. TDP — the thermal design power you must remove — is set by the silicon and the workload. Everything between the junction and the coolant is the resistance stack you must close, within the qualified package, TIM, plate, fluid and flow, and it is governed by a simple, unforgiving relation: the temperature rise from coolant to junction equals the heat removed times the thermal resistance of the path, ΔT = Q × Rθ. Fix Q (the TDP) and Tjunction-max, and the resistance you can afford collapses to a fixed budget. Spend it badly and the chip is illegal at any coolant temperature you can practically supply.

The resistance stack is a series chain — Tjunction to Tcase to the cooling medium — and like any series circuit, the largest resistor dominates. Walk it from the silicon outward, naming the reference temperature beside each resistance. OCP's cold-plate qualification method uses case-to-liquid-inlet resistance: do not add the full coolant rise again to an inlet-referenced result.

The reason air loses at sufficiently high component heat flux is visible in the third row. The convective resistance from a surface to a fluid scales with the fluid's heat-transfer coefficient and the wetted area. Water's volumetric heat capacity is roughly 3,500× that of air, and its convective coefficient at a cold-plate surface is one to two orders of magnitude higher than forced air over a finned heatsink. Air can be pushed harder — more CFM, taller fins, colder supply — but each lever has sharply diminishing returns and a fan-power, noise and pressure penalty that can make further airflow uneconomic. Liquid operates in a different regime rather than merely beating air: it shrinks the case-to-coolant resistor by enough that the same TDP fits inside the same junction budget at a far more relaxed coolant temperature. → the cold-plate engineering is in Chapter 5.4; in-chip microchannels that attack Rθ-JC itself are in Chapter 16.2.

Rack cooling is selected across overlapping equipment and facility envelopes; no universal rack-kW value is a physics cliff. Where earlier Parts speak of a “cooling cliff” or a “density wall”, read both as shorthand for the crossing engineered here — the point where a named rack’s heat split, flux and airflow limits stop closing against the hall you have — and not as a kW threshold. The cited source instead reports 30–40 kW as a typical RDHx range and more than 50 kW with active rear-door fans; those are door-reference conditions, while actual air and liquid-assisted systems close or fail at different duties according to rack airflow and inlet limits, system pressure, containment and recirculation, fan state, acoustics, design-day rejection, service access, and refresh profile. Any published rack, door, or cooling-unit capacity must therefore carry its stated air and water temperatures, flow, pressure, fan state, containment, and heat-capture conditions.

Three physical constraints explain why air becomes unattractive as heat flux and airflow demand rise. First, on a given system curve, fan power rises approximately with the cube of airflow: increasing CFM can impose a steep fan-energy penalty, and the result must be modeled at the actual pressure and fan-efficiency point. Second, air's low heat capacity forces large temperature rises and large volumes; the supply-to-return delta-T air can carry is small, and you run out of mass flow before you run out of fans. Third, acoustic and velocity limits cap how hard you can blow before noise, vibration, and bypass airflow make the hall unworkable and the cooling ineffective at the chip. These constraints make conventional room air unsuitable for the cited 132 kW rack, whose OEM record already assigns ~115 kW to liquid and ~17 kW to air. That product-specific heat split — not subtraction from a universal rack-kW ceiling — establishes its DLC requirement.

Where the supported equipment roadmap may require water at the rack, reserve the hard-to-retrofit structure, routes, isolation, CDU space, and rejection capacity early, even though air, RDHx, hybrid, and DLC envelopes overlap. An inherited air hall may or may not lack those provisions; inspect it rather than assuming. Adding missing liquid and structural interfaces later can run ~$2M/MW (cooling-only) to ~$5–6M/MW and up and still strand power or floor capacity. This is a named equipment-roadmap and facility-envelope decision, not a workload or rack-kW lookup. → retrofit paths are engineered in Chapter 5.10; air-system qualification is in Chapter 5.2.

The density wall would be an academic curiosity if accelerators had stayed put. Per-GPU thermal design power has climbed from the A100's ~300 W to the H100's 700 W to GB200's ~1.0–1.2 kW, with the standard Rubin (VR200) package at ~1.8 kW under Max Q and ~2.3 kW under Max P, while Rubin Ultra has no published per-package TDP. Multiply by the GPUs packed into a rack and the rack-level curve is steeper still, because the scale-up domain grew at the same time, concentrating more silicon behind a single liquid manifold. Across cited reference designs, the transition from H100-class air-capable configurations to the GB200 NVL72's declared liquid/residual heat split narrowed the feasible set from qualified air or RDHx paths to a supported DLC-plus-residual-air architecture.

The rightmost column is the consequence. As density and heat flux rise, the equipment profile narrows the feasible set, but selection still depends on liquid heat fraction, airflow/inlet limits, water availability, TCS/FWS conditions, climate/rejection, residual room heat, serviceability, redundancy, and the future tail. The decisions are when you cross for a named, funded refresh generation and whether the irreversible substrate is ready when you do; reserve infrastructure for those configurations and price later generations as options. What a hall scoped for a previous density cannot make up later is the substrate, not the fit-out — not the floor, not the power chain, not the cooling plant — which is why Chapter 1.1 prices the named ramp scenarios against that substrate before the IT is ordered.

Map the density curve and required thermal interfaces onto the available cooling technologies and you get an overlapping ladder. Air, rear doors, cold plates and immersion each hand the load onward when their pressure, temperature or service envelope runs out. Choose the lowest-cost supported rung that clears peak duty in normal and degraded operation, because every added interface costs capital and service effort you do not get back. Added liquid plumbing spends integration effort, while retaining air spends fan pressure, floor area and inlet margin.

• Air (containment + CRAH/in-row). Saturates when the selected rack’s airflow, pressure and inlet limits run out — a configuration limit, not a universal kW/rack cliff. No facility water at the rack; the simplest service path when the air system closes delivery and rejection at the lowest cost. Still the right answer for air-qualified storage, networking, HGX and RTX PRO nodes, and edge equipment. An NVL72-class rack that requires DLC needs its liquid path for inference as well as training: the rung follows the named rack profile, not the workload label. → Chapter 5.2.
• Rear-door heat exchangers / air-assisted liquid. Bridges the air-cooled rack’s heat into water: passive doors spend server-fan pressure, active doors add their own fans to carry more coil pressure loss. Capacity belongs to the named door’s air/water schedule, not a universal passive or active band. Captures heat at the rack exhaust with a liquid coil; the brownfield-friendly rung when a water-fed door fits the existing server airflow, or a cold-plate L2A unit and room plant carry the full heat without building water. Saturates where the door coil can no longer extract a high enough fraction of the heat. → Chapter 5.3.
• Direct-to-chip liquid (single-phase DLC). A mainstream 2026 path; one reported forecast is ~55%, but PMR's published category is broader cold-plate liquid cooling rather than measured single-phase-DTC deployment share. Cold plates on the GPUs/CPUs/switches, in-rack manifolds, dripless quick-disconnects, a CDU isolating the technology-cooling loop from facility water. Carries the cold-plate path into the 100 kW to 200+ kW rack-design problem when the selected rack, fluid and distribution close; extending warm-water loops to a Kyber-generation roadmap requires that generation’s own heat, pressure and service qualification. → Chapter 5.4; CDU loop in Chapter 5.6.
• Immersion (single- and two-phase). Single-phase keeps the bath below boiling and owes you a wet-server service process; two-phase boils the dielectric and owes you vapor recovery as well as that process. PFAS fluid exposure remains a procurement issue where the named chemistry falls within the applicable definition. Compare auxiliary energy, wet support reactions, fluid recovery and warranty at matched duty before paying for either bath. → Chapter 5.5.
• In-chip / direct-to-silicon microfluidics. The next rung, attacking the in-package Rθ-JC resistor itself with microchannels etched into or onto the die — the only lever that touches the dominant resistance the cold plate cannot reach. Roadmap, not yet default. → Chapter 16.2.

Part 5 leans on a small, consistent vocabulary of thermal metrics. Pin them down here so the engineering chapters can use them without re-deriving:

• Approach temperature — the terminal temperature difference at one end of a counterflow exchanger: the stream leaving on one side measured against the stream entering on the other at that same end. For a CDU running TCS 45 °C in / 35 °C out against FWS 30 °C in / 40 °C out, the cold-end approach is 35 − 30 = 5 K; comparing the two outlets gives −5 K and describes the wrong relationship. Smaller approach, more effective and more expensive exchanger, warmer facility water that still serves the rack. The single knob you tune at every exchanger in the chain.
• Effectiveness (ε) and NTU — effectiveness is actual heat transfer over the thermodynamic maximum; NTU is the dimensionless measure of exchanger size (conductance × area over the minimum heat-capacity rate). The ε-NTU method is how you size CDU and facility heat exchangers without solving the full temperature field. Higher NTU asymptotes toward ε = 1 with diminishing returns.
• Delta-T (ΔT) — the temperature rise a coolant carries across a load. A larger ΔT moves the same heat at lower flow (smaller pumps, smaller pipes), which is why warm-water, high-ΔT design is favored. The flow-rate rule of thumb — ~1.25–2.0 L/min per kW — falls directly out of a roughly 7–12 K rise for water near 1 kg/L and 4.18 kJ/(kg·K). Use the actual fluid density and specific heat for selection, then check the component minimum flow and pressure limits.
• Heat flux (W/cm²) — power per die area, the quantity the cold plate actually fights at the hotspot, distinct from total TDP.

The facility-efficiency metrics — PUE, WUE, ITUE, and TUE — sit one level up, scoring the whole plant rather than a single exchanger. PUE is total facility energy over IT energy; WUE is water consumed per IT energy; ITUE and TUE extend the accounting to capture fan and pump parasitics that liquid cooling reshuffles. These are defined canonically in Chapter 15.1 and used throughout Part 5 as the scorecard for the design choices this chapter sets up; we name them here only so the cross-references resolve.

Trace. IT heat = 4 × 100 = 400 kW; liquid duty = 4 × 80.0 = 320 kW; air duty = 4 × 20.0 = 80.0 kW. Each water stream needs mass flow 320/(4.18 × 10.0) = 7.66 kg/s, or 459 L/min for the zone and 115 L/min per rack. Both heat-capacity rates are 32.0 kW/K. Cold-end approach = 35.0 − 30.0 = 5.0 K; hot-end approach = 45.0 − 40.0 = 5.0 K. Effectiveness = 320/[32.0 × (45.0 − 30.0)] = 0.667. Comparing the two outlets would test the wrong pair.

Component temperature = 35.0 °C + 1,000 W × (0.0150 + 0.0250) K/W = 75.0 °C. Margin is 85.0 − 75.0 = 10.0 K, so the specified 5.0 K reserve remains. Select this operating point for the thermal design and carry the separate 80.0 kW air duty into room cooling; omitting that branch strands equipment outside the cold plates.

Flip. At unchanged resistance and component heat, the reserve is exhausted at 85.0 − 5.0 − 40.0 = 40.0 °C inlet; warmer inlet fails. At fixed TCS temperatures, raising FWS inlet above 30.0 °C also fails the assumed 5.0 K cold-end approach requirement. A different exchanger selection or lower heat load must close the new point. OCP cold-plate qualification §1.3.2.1 defines the inlet reference; the balance remains canonical here. Hand heat and flow to 5.6, external head to 5.13, and installed acceptance to 13.5.

Select the heat path only after the component margin, liquid/air split and both exchanger terminals close at the required load. A warmer operating point spends temperature reserve for rejection opportunity; a colder one spends plant energy for reserve. Carry the selected point into the fluid and hydraulic designs before buying equipment.