Commissioning Fundamentals, Levels & Program Governance

Commissioning (Cx) is the discipline that refuses to take the design's word for it. Every prior part of this guide makes claims — that the power chain will ride through a utility dip, that the cold plates will hold the GPUs under sustained load, that the fabric will carry a non-blocking all-reduce, that the standby plant will pick up the building before the UPS runs flat. Commissioning is the structured process of generating the evidence that those claims are true under the conditions that matter, before the facility carries revenue load. It is the bridge between Part 2's design and Part 14's day-2 operations, and it is the last point in the project where a latent integration defect costs hours to fix rather than a multi-day outage on a live cluster.

For a conventional enterprise data center this is a mature, well-codified ritual. For an AI factory it is not, and the gap is the subject of this entire Part. The reason is structural: an AI facility is two machines pretending to be one. There is the facility — substation, switchgear, generators, UPS/BESS, CDUs, chillers, piping, BMS — which the commissioning industry knows how to accept. And there is the cluster — tens of thousands of GPUs, the scale-up and scale-out fabric, storage, and the scheduler — which the facility commissioning standards barely acknowledge exists. The two tracks run in parallel, share critical resources (power and cooling), and must be interlocked at specific gates, or each track signs off in isolation against a load the other half cannot actually deliver. What follows sets out the levels, the two tracks, the roles, and the governing documents that the rest of Part 13 builds on.

Mission-critical commissioning is organized as a ladder of increasing integration, conventionally Level 1 through Level 5 (with some programs adding a Level 0 design-review and a Level 6 post-occupancy / seasonal stage). Each rung tests a wider boundary than the last; you do not climb to the next rung until the current one is signed and its deficiencies closed. The ladder's rule is that integration faults are found at the lowest level at which they can possibly appear — a mis-wired CT caught at L3 standalone test is a morning's rework; the same fault discovered at L5 during a black-building test invalidates the run and resets days of scripted work.

Freeze the owner’s level names and integration boundaries in the contract; Chapter 13.6 supplies the sole L1–L5 capability ladder. The two parallel tracks (next section) each have their own analogue of this ladder, and Chapter 13.2 turns each rung into a quantitative script with explicit pass/fail gates.

Chapter 13.6 assigns the factory, installation, standalone, discipline and integrated evidence to the L1–L5 ladder. Budget that sequence on both tracks. Uptime Intelligence’s AI in Practice Paper 4: Level 4 & 5 Commissioning (May 2026) describes AI-aware banks on power and liquid loops, scriptable GPU power-versus-time curves, third-party-witnessed L5 and decommissioning procedures written during Cx. Those mechanisms belong in the fixture and witness scope before the schedule is frozen.

Two governance failures are expensive to recover from: deferring a discipline’s own failure tests into the integrated campaign, and truncating that campaign when schedule slips. Require the UPS transfer and battery ride-through evidence before the generator/cooling/BMS handoff test, and name the owner of each dependency in Chapter 13.6’s ladder. A generator can start and a cooling plant can ride through while the BMS still hands the load off in the wrong order. Calling the individual passes “good enough” pushes that integration risk onto day two.

The L5 test matrix is dictated by the resilience claim the facility was designed to meet, and the two claims that matter are the Uptime Institute's concurrent maintainability (Tier III) and fault tolerance (Tier IV). They sound similar and they are not, and commissioning is where the difference becomes physical.

Concurrent maintainability (Tier III) means any single capacity component or distribution path can be removed from service — for planned maintenance — with no impact on the IT load. The IST must therefore prove that you can take down each path in turn, one at a time, deliberately, and the load never notices. It does not require surviving an unplanned simultaneous fault. Fault tolerance (Tier IV) is the stronger claim: the facility survives any single unplanned worst-case failure — including the loss of an entire distribution path to fire or flood — with no load impact, while still being concurrently maintainable. Tier IV adds compartmentalization and continuous cooling requirements that Tier III does not, and the IST must demonstrate the auto-response to a fault, not just an orderly manual switchover.

The consequence for the commissioning program is direct: the resilience target you committed to in design writes your failure-mode demonstration script. A Tier III program scripts orderly, one-path-at-a-time maintenance scenarios. A fault-tolerant-topology program scripts approved, risk-assessed scenarios representing the required fault set, and continuous-cooling ride-through — a materially larger, riskier, longer IST. Choosing the topology is a design and capital decision (Chapter 12.1); commissioning proves each claim with the safest valid method: inspection, secondary injection, hardware-in-the-loop, simulation, staged load banks, or a specifically approved live fault test.

What makes AI-factory commissioning different from every commissioning program before it is structural: there are two acceptance ladders running at once, on different schedules, owned by different organizations, governed by different standards — and they are coupled by shared physical resources.

The facility track (Chapters 13.3–13.6) is the classical mission-critical Cx program: utility energization, switchgear, generators, UPS/BESS, CDUs, chillers, piping, controls, culminating in the L5 IST. It is owned by the owner's CxA and the MEP trades, governed by ASHRAE Guideline 0 and Guideline 1.1 alongside Standard 202's process requirements (the data-center-specific Guideline 1.6P is still a proposal), Uptime, and BICSI 002. The cluster track (Chapters 13.7–13.9) is the IT validation program: fabric commissioning (BER, link-flap, topology, bandwidth/latency), GPU node burn-in and diagnostics, cluster-scale NCCL/collective acceptance, and a reference/proxy training run. It is owned by the platform/SRE/ML-infra team, governed by vendor deployment guides and de-facto standards like SemiAnalysis ClusterMAX — none of which the facility CxA typically touches.

If those two tracks each sign off in isolation, you have certified nothing useful. The facility track accepts a power-and-cooling envelope using load banks — usually air-cooled resistive banks, whose step granularity, control bandwidth and air-side heat rejection set the load they can actually present. The cluster track accepts compute using real GPUs that draw a violent, synchronized, switching load and reject heat into liquid. Neither track, alone, exercises the seam between them. The interlock gates are where that seam is tested.

The middle row is the defining limitation of facility commissioning for AI. A conventional air-cooled resistive bank is a deliberately boring load: elements switched in steps, drawing a steady current at unity power factor and rejecting its heat straight to the room air. What it can and cannot reproduce is set by its step granularity, control bandwidth and heat-rejection path, not by the word 'resistive' — liquid-loop rejection and fast, repetitive waveforms need a bank specified for them. A frontier GPU cluster can exceed that conventional bank's profile — tens of thousands of accelerators that ramp from idle to full and back in milliseconds in lockstep across a synchronous step, imposing power transients and harmonic content that a step-switched bank does not reproduce, and dumping that heat into a liquid loop the load bank never touches. The facility track can prove the power chain is wired correctly and redundant; it cannot prove the chain absorbs the dynamic load swing of a real all-reduce, because the selected air-rejecting, step-switched fixture cannot generate that full waveform; another fixture earns only the coverage its measurements establish. This is the dynamic-load realism gap; its canonical treatment is in Chapter 13.6, the transient physics behind it lives in Chapter 4.5. The mitigation is sequencing: liquid-loop emulation followed by the staged proxy training run of Chapter 13.9 closes the gap, which is why it must come before — not after — go-live.

Commissioning is only as good as the requirements it tests against, and those requirements live in a short chain of governing documents that every acceptance script must trace back to. The chain is deliberately a chain — each document derives from the one before it, so that a pass/fail gate in an L5 script can be followed all the way back to a stated owner intent.

• Owner's Project Requirements (OPR). The owner's intent in measurable terms: the availability target (and therefore the Tier or BICSI class), the density and ramp the building must accommodate, the environmental envelopes, the maintainability expectations, and the acceptance criteria the owner will hold the project to. Everything downstream is an answer to the OPR. For an AI factory the OPR must state the workload intent (training-shaped vs inference-shaped, per Chapter 1.1), because that is what makes the difference between commissioning for goodput and commissioning for availability.
• Basis of Design (BOD). The design engineer's documented explanation of how the proposed systems satisfy each OPR requirement — the topology, the redundancy scheme, the cooling architecture, the setpoints, and the assumptions. ASHRAE Guideline 1.1-2025 covers applying the commissioning process to new HVAC&R systems, and is where the BOD's development and verification is set out; it is not a BOD-only standard. The CxA verifies the BOD answers the OPR before construction, not after.
• Sequence of Operations (SOO). The control-logic specification — exactly how the BMS/EPMS/DCIM is supposed to behave in every normal, maintenance, and failure mode. The SOO is the script-writer's bible: an L4/L5 functional test is, in essence, a line-by-line proof that the real controls match the SOO under real conditions. A vague or incomplete SOO is the most common root cause of a failed IST, because there is nothing precise to test against.

The role that owns this chain is the Commissioning Authority (CxA) — and the governance decision that matters most is to engage the CxA at the design phase, as an independent party, not as a general-contractor self-check bolted on at the end. ASHRAE Guideline 0 (the process), ASHRAE Standard 202 (the formalized commissioning process / Cx-Process), and the proposed data-center-specific Guideline 1.6P — on ASHRAE's register as a proposed guideline, not a published one — provide process guidance with different scopes; this guide recommends an independent CxA verifying the OPR→BOD→SOO chain across the whole project lifecycle, from design review (L0) through post-occupancy. An owner who engages the CxA only to 'witness the IST' has already lost the design-phase reviews where the cheapest defects are caught.

Give each interlock an evidence owner and a release owner. The electrical lead supplies bus waveforms and relay events; the mechanical lead supplies branch flow, temperature, pressure and actual protective response; the platform lead supplies device health, workload correctness and recovery. The CxA joins those records against OPR/BOD/SOO requirements; operations accepts the restored state and the owner names the person authorized to release load. These are example responsibilities, not signatures. Record the installed rack interfaces, utility boundary and OT access/restore obligations from Chapter 11.11 in the same applicability register. A missing vendor result stays open under its named owner instead of disappearing between two green dashboards.

A conventional data-center commissioning program is a single, well-standardized ladder culminating in an IST against load banks, governed end-to-end by ASHRAE/Uptime/BICSI, signed by an independent CxA. That program, applied unchanged to an AI factory, certifies a building that has never seen its actual load and a cluster the standards never mention.

The three structural breaks are: (1) two tracks, not one — facility and cluster, on different schedules and standards, requiring explicit interlock gates; (2) load realism — the air-rejecting, steady load banks in a conventional scope cannot reproduce the dynamic power and liquid-side thermal behavior of real GPUs; switched and liquid fixtures extend that coverage, so a staged, product-representative run closes workload-specific normal-operation evidence that facility emulators do not cover; its scope and acceptance criteria follow the approved project risk and test plan; and (3) density ramp — GB300 NVL72 is deploying through 2026 alongside GB200, Vera Rubin entered full production in August 2026, and the announced ~600 kW Rubin Ultra/Kyber planning point follows in H2 2027, so the IST must validate not only the steady state but the headroom the ramp will consume (floor, water, electrical, cooling-plant turndown). A program that handles these three is a Part 13 program; one that ignores them hands a long deficiency list to day-2 operations — exactly the failure environment the 419-interruptions-in-54-days reality of a real training run cannot afford.

Choose the interlocked program whose required states have a valid test method, an evidence owner and a funded slot in the schedule. Finishing facility and cluster checklists separately is cheaper to organize, but the first synchronized workload then becomes an unscheduled test of their shared power, cooling and controls.