The Archetype Decision Framework: Workload Is the Master Variable

Every other decision in this guide flows from one question that is almost never asked first: what is this machine for? The voltage you step down to, the water temperature at the rack, the back-end fabric's oversubscription ratio, the redundancy tier and the county you site in all sit downstream of it. Not "AI" in the abstract: the specific workload, its coupling, its tolerance for interruption, and how close it must sit to the user. Answer that well and the rest of the lifecycle is a series of well-posed engineering problems. Answer it badly and you have spent two to four years and a capital stack on a facility mis-matched to the revenue it was supposed to earn. A change order will not fix the worst of those mistakes: they are undone only by moving, re-applying for utility service, reworking the floor structure, or tearing out plant under a running hall.

This chapter forces the question to the front. It defines the five workload archetypes and the four procurement archetypes. It then traces the requirements cascade, by which the workload, the hardware you choose and your uptime and latency targets set density, cooling, fabric, storage, redundancy, siting and TCO, and sorts those decisions into reversible and irreversible.

Scoping an AI data center means answering two orthogonal questions: you must answer both, and answering one does not settle the other. WHAT will run here: pre-training, post-training/RL, online inference, batch inference, or edge inference. In practice it is a weighted mix, but one dominant archetype sets the design basis. HOW will the capacity be procured and built: greenfield self-build, retrofit of an existing hall, wholesale or retail colocation, or rental from a GPU neocloud. A frontier pre-training run or an online-inference business can live in any of the four. The cross-product is a 5x4 matrix of real, deployed facilities, and the cost of mismatching a cell to its workload is the recurring theme of Part 1.

This matters more in 2026 than in 2020 because the binding constraint moved. The industry was chip-bound, limited by how many accelerators it could buy. It is now power-bound, limited by how many megawatts it can energize, and when. Microsoft's CEO said as much in November 2025: the company may have “a bunch of chips sitting in inventory that I can't plug in” for want of powered shells, and its CFO had told analysts the constraint was space and power, not chips. The US generator interconnection queue held more than 2,060 GW of active generation and storage capacity at the end of 2025, over one and a half times the entire US installed fleet. The median request-to-COD time was around five years for the generation and storage projects built in 2025 (LBNL). That queue is generation and storage supply, not data-center load requests; your energization date comes from your host utility's tariff, studies, upgrades, agreements and milestones. When power is the scarce input, a mis-scoped archetype wastes capital and burns an interconnection slot you cannot get back, against a depreciation clock that is already running.

Three properties distinguish the five archetypes and drive everything else: coupling (how tightly the accelerators must communicate within one unit of work), interruption tolerance (whether the job survives a node failure cheaply or restarts from a checkpoint), and latency sensitivity (whether a user is waiting on the output in real time). These three levers translate "what runs here" into "how it must be built."

Pre-training is one tightly-coupled supercomputer. Thousands of GPUs run synchronous data, tensor, pipeline and expert parallelism, with all-reduce/all-gather collectives crossing the back-end fabric on every step. The whole job moves at the speed of its slowest straggler. In a static synchronous run, one failed GPU forces a restart from the last checkpoint; elastic-membership and checkpointless implementations soften that, at the price of replicated state, spare capacity and a tested recovery path. Pre-training pushes hardest on density, direct-to-chip liquid capture, back-end bisection and scale-up domain size, though your rack's heat split, measured collective traffic and step-time target set each one, not the label. → Chapter 1.2.

Post-training / SFT / RLHF / RL is the hybrid middle, and the most misunderstood of the five. Supervised fine-tuning is a small, bursty training job. Large-scale RL for reasoning, though, is inference-heavy training: generating rollouts costs more than the gradient update, so the cluster looks like a fleet of inference engines feeding a comparatively small trainer (deep dive below). Scope it as pre-training and you over-provision the fabric; scope it as pure inference and you starve the policy update. → Chapter 1.4.

Online (interactive) inference is the revenue workload for most operators: bursty, latency-bound and always-on. Traffic can swing from 30% to 90% of capacity in minutes, and an SLO on time-to-first-token and time-per-output-token governs the user experience. Most requests fit inside one node or a small scale-up domain, so the back-end fabric can be oversubscribed, but the facility must be highly available, and close to users where the serving path's latency budget requires it. Reasoning models emit long decode sequences, which has inflated the decode share and KV-cache pressure and reshaped fleet sizing. → Chapter 1.3.

Batch (offline) inference covers embeddings generation, document processing, synthetic-data creation and evaluation sweeps. It is throughput-bound, not latency-bound. No user is waiting, so it tolerates interruption, queuing and aggressive oversubscription, and it is the natural consumer of spot capacity, off-peak power and curtailable interconnections. It is the cheapest archetype to host and the most flexible to schedule. It is also the only one without its own chapter, because its requirements relax the serving case rather than add new ones (see Chapter 1.7 and Chapter 14.7).

Edge inference pushes serving to the user: on-prem appliances, telco/MEC nodes, Tier-2 metro colos, CDN-adjacent sites. A hard latency budget defines it (the 30/50/100 ms perceptibility thresholds), along with severe power, thermal and space limits, and it runs lights-out with zero-touch provisioning. Its siting driver is the inverse of pre-training: physical proximity, not cheap power. → Chapter 1.5.

Read the table as a cascade. Archetype and coupling are starting descriptions; your equipment, measured traffic, site limits and service tests turn the other columns into a design basis. For pre-training on the cited NVL72 equipment, what you still have to prove out is liquid cooling, a non-blocking fabric, a floor for the rack's mass and a power-first site search. Today's wet racks weigh ~3,000–3,300 lb (HPE's filled GB200 NVL72 is 3,245 lb), and NVIDIA describes the Vera Rubin NVL72 as roughly 4,000 lb, so carry a contingency for heavier racks; Chapter 6.2 works a ~3,000–5,000 lb planning envelope. If the serving path cannot meet its latency budget from that site, online inference needs a latency-first site that usually costs more on energy (price the delivered tariff; see Chapter 1.3), with fabric and KV-cache tiers sized to tail latency rather than step time. A fault-tolerant power path follows from a requirement that maintenance or a single component fault must not interrupt service, not from the workload name. The workload ties a dozen subsystem decisions together, but measurements settle them.

The coupling reaches past the subsystems into the building. Microsoft built its Fairwater AI sites two-story so racks could be placed in three dimensions to shorten cable runs, and linked Atlanta to its Wisconsin site over a dedicated AI WAN that extends the training fabric across sites (Microsoft, 2025). → Chapter 6.2 (multi-story structure), Chapter 8.8 (scale-across).

The second question, how you acquire the capacity, trades four levers against each other: time-to-power, capital intensity, control and workload duration. The right answer depends on how long you expect the workload to run and how sure you are of it. Compare speed, control and unit cost for the same scope of service; no procurement label guarantees the ranking.

Greenfield self-build gives the most control over density, power architecture and cooling, and possibly a lower long-run unit cost if demand is large and firm enough. It costs the longest, queue-gated schedule to a live cluster and the deepest capital commitment, so it suits only a durable, well-forecast workload at scale. Retrofit of an existing air-cooled hall saves capital and schedule but hits a hard physics ceiling: floor loading, plenum, electrical headroom and available water cap how far density can go. Colocation (wholesale or retail) buys time-to-power by renting someone else's powered shell: a live 50k+ GPU cluster in a wholesale hall on the provider's schedule, not the grid's. Neocloud / GPU rental cuts time-to-first-job to however fast a provider can hand you capacity and converts capex into opex, at the contracted per-GPU-hour rate and with only the contracted control over fabric and reliability. The decision is rarely all-or-nothing: hybrids (burst-to-neocloud, colo-anchor-plus-cloud-overflow, build-core-rent-edge) are the norm. → Chapter 1.6; quantitative NPV in Chapter 1.8.

The cascade is a causal chain you can walk forward from a single input: density → cooling → fabric → storage → redundancy → siting → TCO, each link constraining the next.

Density sets cooling. This is the hardest link because it is physics, not policy. The cited HPE GB200 NVL72 record assigns roughly 115 kW to liquid and 17 kW to residual air. That rack's heat split and airflow and inlet limits, not a universal rack-kW cliff, make direct-to-chip liquid necessary for it. The GB200 DLC range is tight but warm-water-capable: the rack accepts supply up to ~45 °C and return up to ~65 °C. Those are acceptance maxima, not the rise you size flow to. The ~165–236 L/min guide flow band is a separate water-property heat balance for a 7–10 °C design rise across the ~115 kW liquid share; Clariant PG25 at the same rise gives ~172–246 L/min, roughly 4% more. Warm water is the design intent; colder supply buys thermal headroom at the cost of chiller capex. The density target therefore commits the cooling plant and heat-rejection strategy. → Chapter 5.1 (the density wall) and Chapter 5.4 (DLC).

Measured coupling sets the fabric. Published designs span 1:1, 2:1–3:1 and reported 7:1 examples, but those are observations, not training or inference defaults. Derive each tier's ratio from the measured traffic matrix, collective and request mix, placement and communication overlap, topology, failure headroom, and step-time or tail-latency SLO, then validate it on the target fabric. Synchronous collectives often justify high bisection. Local inference often permits upper-tier oversubscription, while distributed MoE, KV movement and prefill/decode disaggregation can require more. Scale-up domain size (8 GPUs in HGX, 72 in NVL72, 144 packages / 576 dies in the announced Kyber rack) likewise follows the parallelism and failure-domain plan. → Chapter 8.5 (topology & oversubscription).

Interruption tolerance sets redundancy. For a synchronous training job, checkpoint-and-resume makes a single-path or component-redundant design viable as long as the work lost to a restart stays within the job's completion target. Microsoft's Fairwater Atlanta training site shows this at scale: it forgoes on-site generation, UPS and dual-corded distribution for the GPU fleet and relies on a highly available grid feed, reporting “4×9 availability at 3×9 cost” (Microsoft, Nov 2025). That works because the jobs checkpoint and the grid feed was proven first; it is not a template for an always-on serving hall. An always-on inference business, facing lost revenue and SLA exposure, often cannot drop load during maintenance or a single component fault. That requirement, not the workload name, is what justifies a stronger topology. The cleanest anti-pattern here is over-provisioned redundancy for checkpointable jobs: it buys nines the workload does not value. → Chapter 12.2 reframes this as goodput vs availability.

Latency sensitivity sets siting. Pre-training does not care about user proximity, so it chases the cheapest firm megawatts it can energize soonest, even hours from any metro. With warm-water liquid cooling, climate is a secondary term: it still sizes residual air, heat rejection and water use, but the flagship training campuses in Georgia, Texas and Louisiana show it no longer picks the site. Online and edge inference invert this when the measured serving path misses its latency budget from a cheaper site. Fiber adds about 1 ms of round trip per 100 km, and under load, queueing and prefill often outweigh network distance in time-to-first-token, so test the path before paying for proximity. Where proximity does bind, price the premium from delivered tariffs. As a first screen, EIA's January–July 2026 state industrial averages run about 1.5x higher in Virginia, Illinois, Ohio or New York than in Texas or Louisiana (about 1.2x in Georgia), and about 2.3x in New Jersey and 3.1x in California. Chapter 1.3 sets the rule for when proximity is worth paying for. Siting is the least reversible decision of all (you cannot move a slab), so derive it from the workload, never the other way around. → Chapter 3.1 (the reordered siting hierarchy) and Chapter 3.2 (speed-to-power).

Not all forks cost the same to re-decide. Sort decisions by the cost of changing your mind and spend your optionality budget to match. Price each irreversible reserve against the later retrofit and outage cost, and defer a reversible choice only while migrating later stays affordable.

Irreversible (decide once, at scoping): the site itself (you cannot move a slab); the grid interconnection capacity and voltage class (the queue slot is the scarcest asset in the project); the structural floor-loading basis (re-engineering a floor for 3,000+ lb wet racks mid-life is brutal); the base power architecture (415/480 VAC vs an 800 VDC path); and the macro cooling decision (a hall plumbed for liquid or not). This is where you pay to keep options open, say by provisioning floor loading and water for a density step-up you have not committed to.

Reversible (defer, re-decide cheaply): the accelerator generation within a given power and cooling budget; the scheduler and orchestration platform; the oversubscription ratio on a fabric you sized non-blocking; the workload mix within an archetype; and, critically, the procurement mode, which is why hybrid and rental exist. The strategic move is to turn irreversible decisions into reversible ones wherever the option premium is cheap. A powered shell instead of a full build-to-suit preserves IT-fit-out optionality. A colo lease preserves only the termination, assignment or break rights you negotiated. Reserving floor loading and water headroom preserves a density ramp. → procurement framing in Chapter 1.6; refresh execution in Chapter 14.9.

A scope that survives board scrutiny and lender diligence is three artifacts that pin down the archetype decision and its consequences, all signed before long-lead equipment is ordered.

• Workload profile sheet. The dominant archetype and mix ratio; coupling, interruption tolerance and latency budget; the validated scale-up domain size, GPU:CPU and GPU:storage ratios, and the back-end fabric blocking requirement. The whole cascade is derived from this one page.
• Capacity ramp curve. MW and GPU count over time, generation by generation, with the density step-ups called out. Mark funded reserves apart from forecast, because floor, power, water and cooling commitments each need an explicit expansion trigger.
• Design-basis document. The frozen assumptions everything downstream inherits: density tier, cooling modality, redundancy topology, voltage architecture, siting class, and the reversible-vs-irreversible register of which assumptions are hedged and which are committed.

Per-archetype reference design-basis sheets and scalable-unit budgets are in Chapter 1.7; the economics that score the scope are in Chapter 1.8.

The same mis-scopes keep recurring, because each skips the archetype question and reasons from the equipment or the real estate instead. Three are worth naming:

• Sizing fabric from the workload label. SemiAnalysis’s October 2024 1,024-H100 training-cluster model estimates 31.6% back-end savings from combining virtual modular switch placement with 2:1 oversubscription against NVIDIA’s reference architecture; the non-blocking virtual modular switch alone saves 24.9%. Neither 1:1 nor 2:1 is a training or inference default. Some node-local serving profiles may validate upper-tier oversubscription; distributed MoE, KV movement or prefill/decode disaggregation may not. Derive each tier from measured traffic as above, and spend only validated savings on geo-distribution and uptime.
• Forcing a rack beyond the cooling the hall inherited. Trying to land 132 kW liquid-cooled racks in a hall scoped for 40 kW air. The power chain, the plenum and the existing heat-rejection path say no, and often the floor does too. A retrofit fails or strands capacity when any interface the rack needs stays out of reach, so compare a fully scoped conversion with a purpose-built liquid hall before choosing.
• Choosing topology from checkpointability alone. Checkpoint recovery changes what an interruption costs, not which maintenance or fault events may drop the hall. Before choosing N, N+1, distributed or duplicated power, define transfer interruption, post-event loading, path and control independence, common modes, the recovery SLO and the customer contract. A topology is over-provided only if that event-by-event comparison shows a cheaper path meeting the same target. → Chapter 12.2.