Orientation: The AI Data Center as a Single Co-Designed Machine

A traditional data center could treat compute, fabric, power, cooling and software as separate disciplines with separate vendors and org charts, because a 10 kW rack of web servers is loosely coupled to its building. A 132 kW liquid-cooled rack of accelerators running a synchronous training job is welded to its power chain, water loop and fabric by physics. A decision in any one of them propagates into the others within milliseconds, and across the entire capital stack.

Scope the facility as separate subsystems, each optimized by a different team to a different metric, and the result is predictable. The power chain is sized for a density the cooling plant cannot remove. The fabric is non-blocking for a workload that never needed it. The redundancy is gold-plated without pricing what checkpointed jobs lose during recovery. The PUE is excellent and the goodput mediocre. Every subsystem passes its own acceptance test, yet the machine can still underperform its TCO model.

This chapter sets up the lens for the sixteen Parts ahead: co-design, one objective function, two interlocking tracks and two narrative spines. It is short on numbers by design; the derivations live in the chapters it points to. Before the first subsystem order, write down the accepted workload output, the service limit and the cost horizon.

An AI factory is graded on a system objective: TCO per unit of useful work. That is the lifetime cost of the asset, energy included, divided by the useful work it delivers over the same economic horizon (training steps that survive, tokens that meet their SLO). Almost every subsystem metric is a proxy for it, and every proxy lies in a different direction. PUE rewards spending less energy on cooling, yet the lowest-PUE choice can throttle the GPUs and destroy more goodput than it saves in overhead. Rack fill rewards dense packing, but past the cooling cliff it strands power you paid to interconnect. Switch utilization rewards oversubscribing the fabric, and on a synchronous training job that collapses model-FLOPs-utilization while the whole cluster idles waiting on a straggler. Each local optimum is a concrete number a team can be measured against, and each, pursued alone, can move the system objective the wrong way.

Co-design means the forks in this guide are coupled. The voltage you step down to, the temperature of the water you push to the cold plate, the oversubscription ratio on the back-end, the redundancy tier you commission and the county you site in cannot be decided in isolation without costing the others. Treat the design as a constrained optimization with one objective and many coupled variables, not a checklist of independently graded subsystems. When you read a decision in a later chapter, ask not "is this subsystem efficient?" but "does this choice raise system goodput per dollar, net of what it costs the subsystems it touches?" → the metric stack that makes this measurable is defined in Chapter 0.3 and measured in Chapter 14.1, with efficiency definitions in Chapter 15.1; the reliability reframing of goodput over availability is in Chapter 12.2.

The co-design thesis was already practice in HPC and hyperscale in 2020, since liquid-cooled supercomputers and warehouse-scale facility/IT co-design both predate this cycle. In the typical enterprise hall, though, racks drew 10–15 kW, air cooling was the default, and the binding constraint was how many accelerators you could buy, so co-design stayed academic. Three things broke at once between 2024 and 2026 and turned it from good practice into survival.

First, density exploded. A 2023 four-system DGX H100 rack requires 40.8 kW. The HPE GB200 NVL72 has a 132 kW nominal rack TDP. GB300 NVL72 shipped in 2025 and is deploying through 2026 alongside GB200; Lenovo rates it at 135 kW TDP and up to 155 kW peak, and NVIDIA uses an up-to-142 kW full-rack facility basis. The Vera Rubin VR200 NVL72 has been shipping since June 2026: Pegatron rates it at 188 kW Max Q and 228 kW Max P, and NVIDIA DSX sets a 330 kW cabinet facility design basis. NVIDIA's roadmap sets a ~600 kW Rubin Ultra / Kyber planning point on an 800 VDC path, targeted for H2 2027. SemiAnalysis reported in July 2026 that Kyber slips to 2028, which NVIDIA disputes, so do not schedule against it (NVIDIA DGX H100 design guide; HPE QuickSpecs; Lenovo Press LP2357; NVIDIA Enterprise RA and DSX Facilities Infrastructure Design Guide v2.0; Pegatron RA4803-72N3; NVIDIA GTC 2025 keynote and 800 VDC architecture post). That is a roughly 15x endpoint comparison from the 2023 DGX H100 rack to the ~600 kW Kyber roadmap point, across different rack profiles and power bases; it is not a measured fleet-wide ramp. This trajectory outruns conventional room-air designs. Choose cooling from the rack's airflow and inlet limits, liquid heat-capture fraction, residual room heat, TCS/FWS availability, climate, redundancy, serviceability and future-density range. That choice rewrites the building, the floor loading and the water strategy.

Second, the bottleneck moved from the chip to the substation. Where energization gates the phase, the project is power-bound: its scarce input is energizable megawatts and the time to get them. Host-utility large-load service can be the schedule gate, and, unlike the generation side's public interconnection-queue datasets, the United States has no national load-queue dataset or portable end-to-end range. Build the date from the site's tariff, studies, facilities/network upgrades, agreement milestones and equipment path. US substation-transformer lead times exceeded 160 weeks in Wood Mackenzie's 2026 assessment (reported by Data Center Knowledge, May 2026) and can dominate the energization schedule. When power and its long-lead equipment are the constraint, read the whole design backward from the point of interconnection, not forward from the GPU. That reverses the traditional design order, and Part 3 and Part 4 make it concrete.

Third, the forecast workload mix shifts toward inference. Deloitte forecast in November 2025 that inference would take roughly two-thirds of AI compute in 2026, up from about half in 2025 and a third in 2023 (Deloitte TMT Predictions 2026). The build-out's center of gravity is moving from a few enormous synchronous training machines toward many always-on, latency-bound, geo-distributed serving fleets. That is a different machine with a different objective, which is why the archetype question comes first, in Chapter 1.1.

The single machine is delivered by two tracks that must interlock, and most failures live in the seam between them. The facility track owns the grid-to-chip power spine and the chip-to-atmosphere heat spine: utility interconnection, on-site substation, MV/LV distribution, UPS and backup generation, the cooling plant, the CDUs and water loops, and the building shell around all of it. The IT/cluster track owns the compute and the software that turns it into a computer: the accelerators and their racks, the scale-up and scale-out fabric, the storage hierarchy that keeps the GPUs fed, and the scheduler and orchestration stack that decides what runs where. Traditionally these were two organizations with two budgets, two vocabularies and two acceptance regimes, which met at the rack PDU and otherwise ignored each other.

In an AI factory that division is the defect. The fabric topology (IT) dictates rack adjacency, and so the floor plan and the water manifold layout (facility). The accelerator's coolant inlet spec (IT) dictates the warm-water loop delta-T and the heat-rejection plant (facility). The contracted maintenance and fault states, interruption limits, post-event loading, recovery SLO and workload consequence decide whether a duplicated power path creates value; checkpointing changes the cost of an outage but does not select the topology. The capacity ramp curve (IT, generation by generation) sets the floor loading and water headroom the facility must provision irreversibly, years before the racks arrive. Each of these sentences starts in one track and ends in the other.

An owner's organization that gives nobody authority across the seam discovers the disagreements at integration, where they are most expensive and least reversible. The fix is a single technical owner of the system objective, with both tracks reporting into it. → the owner's organization and delivery models are in Chapter 2.2; the integrated master schedule that forces the two tracks onto one critical path is in Chapter 2.1; the rack as the integration unit where the two tracks physically meet is in Chapter 7.13.

Because the machine is continuous, this guide tracks two physical chains end to end: Parts 3 and 4 walk the first, Part 5 walks the second, and every Part after them inherits a constraint from one or the other. They are the two flows the building exists to manage: electrons in, heat out.

The grid-to-chip power spine follows a single electron from the point of interconnection to the transistor: utility substation → on-site substation and MV distribution → transformation and switchgear → UPS / ride-through → power distribution → busway and rack PDU → the power shelf and the on-package voltage regulators that finally deliver sub-volt power to the accelerator die. Every transformation in that chain has an efficiency and a failure mode. The cumulative loss is the gap between the megawatts you pay the utility for and the megawatts that reach the chips, and the case for re-architecting this spine is the headline of the 2026 era. For the guide's four assumed AC conversion stages, 0.99 transformer × (0.94–0.97) UPS × 0.96 PSU × (0.90–0.94) VRM yields 80.4–86.7% utility-to-VRM; 0.99 × 0.94 × 0.96 × 0.92 = 82.2% is the representative fixture. Any LV distribution or busway loss is a separate explicit multiplier. A separately sourced SST/800 VDC architecture model is ~87% (SemiAnalysis, May 26, 2026). Compare it with the AC fixture without hiding another distribution stage inside either boundary. → the spine is engineered across Chapter 4.1 (topology and voltage), Chapter 4.2 (interconnect and MV distribution) and Chapter 7.12 (on-package delivery); the power-bound era that frames it is Chapter 16.1.

The chip-to-atmosphere heat spine follows a single joule the other way: the accelerator die → thermal interface → cold plate → in-rack manifold and quick-disconnects → CDU (which isolates the technology-cooling loop from facility water) → facility water loop → heat rejection (chillers, dry coolers, towers, adiabatic, or economizers) → and finally the atmosphere or a heat-reuse offtake. Nearly all the energy that enters the power spine leaves through the heat spine, so the building is, thermodynamically, a machine for moving that heat. The 2026 density climb forces this spine from air to direct-to-chip liquid. Each interface in it (the coolant inlet temperature, the loop delta-T, the approach temperature at the rejection stage) ties the accelerator at one end to the site's climate at the other. → the spine is engineered across Chapter 5.1 (the density wall), Chapter 5.4 (DLC), Chapter 5.6 (CDUs), Chapter 5.7 (warm-water loops) and Chapter 5.8 (heat rejection).

The guide follows the project lifecycle, because that is the order in which the co-design decisions are made and the order in which they become irreversible. Reading it front to back walks the machine from intent to operation; reading it by discipline or by workload archetype works equally well, supported by the cross-references. The spine is: Strategy (Part 1, what machine and why) → Delivery (Part 2, how it gets built and financed) → Siting (Part 3, where, read backward from power) → Power and Electrical (Parts 3–4, the grid-to-chip spine) → Cooling (Part 5, the chip-to-atmosphere spine) → Building (Part 6, the shell that houses both) → Compute (Part 7) → Networking (Part 8) → Storage (Part 9, the IT track that turns chips into a computer) → Software (Part 10, the scheduler that runs the machine) → Security & Reliability (Parts 11–12) → Commissioning (Part 13, proving the machine before it earns) → Operations (Part 14, running it for goodput) → Sustainability (Part 15) → Future (Part 16).

Each concept is derived once, in its canonical chapter, and referenced everywhere else; a roadmap forward-pointer is a bullet, never a chapter. This orientation installs the lens (one machine, one objective, two tracks, two spines, power-bound), and the engineering starts in the Parts that follow. Chapter 0.2 explains how to read the decision-and-consequence framing that structures every later chapter. Chapter 0.3 covers the vocabulary and metric stack you will need throughout. Chapter 1.1 is the workload-archetype decision, the first real fork downstream of this orientation.

Three threads recur in every Part and are flagged at the top of each chapter, so you can read the guide along any of them. POWER-BOUND tracks the phases where megawatts and time-to-power are the binding constraint. It reorders siting, dominates the schedule, and makes the grid-to-chip spine the central narrative of Parts 3 and 4. GOODPUT is useful work delivered within the declared quality and service limits; dividing it by lifetime TCO gives the economic objective, which moves reliability (Part 12), operations (Part 14) and efficiency (Part 15) off subsystem metrics and onto the one number that pays the capital stack. DENSITY-RAMP tracks the move from 10 kW racks toward published roadmaps reaching 600 kW per rack. It drives the chip-to-atmosphere spine across the cooling cliff, sets the irreversible building substrate, and forces a price on each future density option before anyone commits to the reserve. The three couple: density-ramp drives the heat spine, power-bound constrains the electron spine, and goodput adjudicates between them.