Reference Designs & Worked Examples

This appendix is the reusable arithmetic layer behind Part 1's archetype framework (Chapter 1.1) and Part 1's requirements matrix (Chapter 1.7). It does four things, in order: (1) a per-archetype design-basis sheet that freezes the inputs every later number inherits; (2) a scalable-unit (SU) budget giving the power, cooling, water, and network draw of one atomic deployment block per accelerator generation; (3) a 50 MW campus sized from the SU up, with the power chain, cooling plant, and water loop derived; and (4) a 100k-GPU cluster reference BOM with counts and costs for GPUs, racks, CDUs, switches, optics and storage: an August 2025 GB200 estimate, 2026 assumptions and unquoted networking.

The method throughout is multiplier-first. A reference design is a chain of ratios: GPUs per rack, racks per SU, kW per rack, L/min per kW, NICs per node, optics per NIC, GB/s per GPU of storage. Once those are pinned, rack aggregates are multiplications you can audit; shared plant and fabric tiers need the explicit ledgers below. Counts in the tables are exact arithmetic from the stated ratios. Dated server estimates and assumed allowances need project quotes before budget release. Density, flow, and current figures are generation-stamped because they step discontinuously across GB200 → GB300 → Vera Rubin → Rubin Ultra Kyber; do not interpolate across a generation boundary.

The design-basis sheet is the single page that everything downstream inherits — the concrete instantiation of the workload-profile and design-basis artifacts named in Chapter 1.1. Choose the column that matches your dominant archetype, and the rest of the appendix is parameterized for you. The two reference builds in §3–§4 use the frontier-training column unless noted, because it is the most constraining; an inference-shaped build has its own basis: enterprise HGX relaxes density and fabric, while frontier serving runs the same NVL72-class racks and often buys more service redundancy, not less.

The scalable unit is the atomic deployment and costing block — order it, integrate it at the factory (L11/L12), ship it, energize it, repeat. Sizing the SU once and then multiplying is what makes campus and cluster arithmetic tractable. We anchor the SU to the NVIDIA DGX SuperPOD GB200 reference: 8 × NVL72 racks = 576 GPUs per SU, with the full SuperPOD at 16 SUs (128 racks, 9,216 GPUs). The budget below gives one SU's draw across four generations; later sections count SUs, not racks.

The cross-generation columns exist because the multipliers step. A GB200 SU is ~1.06 MW of IT; the same 8-rack SU at Kyber density (~600 kW/rack) is ~4.8 MW — a 4.5× jump in what eight racks draw; qualify the Kyber rack's dimensions, mass, and service clearances before assuming it lands on the same footprint. This is the density-ramp trap from Chapter 1.1 as a budget line: the floor, water, and busbar you reserve today must survive it.

Now multiply. The brief: a 50 MW-class IT frontier-training reference campus on HPE GB200 NVL72, built as integer SUs, with the power chain, cooling plant, and water loop derived. We size the IT budget on rack TDP and the electrical chain on each platform's published peak and OEM facility design basis (HPE’s printed GB200 peak conflicts with its stated multiplier; obtain the corrected duty and duration schedule), and we reserve floor/water/busbar headroom for a GB300 → Vera Rubin density step (the irreversible substrate from Chapter 1.1).

SU count. Choose 48 SUs as a clean reference arrangement: three 16-SU SuperPOD-scale halls, or six 8-SU halls. That is 48 × 8 = 384 NVL72 racks and 48 × 576 = 27,648 GPUs. Treating 132 kW/rack as an exact nominal cap fixture, total IT is 50.688 MW of GPU-rack IT, which exceeds a hard 50.000 MW cap by 0.688 MW. If 50.000 MW is a hard cap, use at most 47 SUs: 376 racks, 27,072 GPUs, and 49.632 MW of GPU-rack IT. Supporting network, storage and management IT still have to fit: Chapter 4.1 closes the total-load ledger; Chapter 5.1 carries every residual-air and liquid duty into the plant. Neither rack count proves total-IT admission.

Campus port map. In each rail/pod, leaf ports 1–32 face hosts and 33–64 face spines 1–32. Each spine uses 1–27 for leaves, 28–32 dark, 33–48/49–64 for its two group cores. Each core’s four consecutive 16-port groups face the four pods’ matching spine. Every cable end has one address. The ledger totals 27,648 host + 27,648 leaf–spine + 32,768 spine–core = 88,064 channels, 176,128 module ends and 704,512 active fiber paths; surveyed lengths determine fiber-kilometers, and electrical lanes are separate.

At 108 leaves/rail the campus exceeds Chapter 8.5’s 64-leaf two-tier limit. Include the core; at ≤16,384 GPUs its two-tier instance becomes eligible. Release only after Chapter 13.7/Chapter 13.9 pass the purchased stack and failure states. A spare chassis repairs an outage; it cannot keep a single-attached rank connected.

The flagship build: a 100,000-GPU GB200 reference training cluster, costed as a bill of materials. Built from the SU: ceil(100,000 ÷ 576) = 174 SUs minimum; choose eleven 16-SU halls, hence 176 SUs = 1,408 NVL72 racks = 101,376 GPUs (≈ 100k). At 132 kW/rack that is ~186 MW GPU-rack IT and ~220 MW facility for that fixture at assumed PUE 1.2. Add separately counted supporting IT before a project capacity release. That capacity may be delivered on one qualified campus or split across sites; DCI does not imply that this illustrative logical cluster runs as one cross-site synchronous job (Chapter 8.8).

The figures use SemiAnalysis’s August 2025 server-only estimate and explicitly assumed storage/CDU/plant costs. Use the counts as arithmetic — they balance ports, power, and flow; they do not draw a topology — and the dollars as an order-of-magnitude frame. The GPU/rack line is a major cost; its share requires priced networking and reconciled facility scope. A percentage allowance cannot supply the core.

Large-cluster port map. Leaves keep the campus assignment. In each rail, twelve pods use spine ports 1–32 for leaves; the final pod uses 1–12, with 13–32 dark. Spine ports 33–64 face eight group cores, four ports/core. Each core’s thirteen four-port pod groups occupy 1–52; 53–64 stay dark. This maps 101,376 endpoints to 8,544 switches; all channel/module populations are in the BOM. It is a realizable map, not a minimum-switch proof.

A campus core loss removes half its group’s upper capacity; a cluster core loss removes one eighth. A leaf loss disconnects 32 ranks. Test these cuts under Chapter 8.5 with actual placement and payload service; healthy port balance does not accept a failure. Budget release stays HOLD for route, recovery, power and quote evidence. At ≤16,384 GPUs, qualifying the two-tier map can remove the core purchase.

Choose the reference population whose counted fabric and surviving workload fit the project; procure the complete scope. 8.5 changes topology; 1.8 values it. An incomplete fabric allowance leaves required hardware, power and commissioning outside the purchase. Close the supporting-IT inventory with 7.13 and the counted fabric in 8.5; then 4.1 owns electrical demand and surviving source capacity, while 5.11 owns cooling flow and heat rejection with a train unavailable. The integrated release needs the actual storage, management and network load, qualified failure state and non-overlapping quotes; rack nominal MW alone releases none of those interfaces.