Metering, Power Quality, Monitoring & Electrical Operations

Everything upstream of this chapter built the electrical machine: the substation (Chapter 4.2), the transformers and harmonic mitigation (Chapter 4.4), the UPS and storage spine that absorbs transients (Chapter 4.5), the DC distribution revolution (Chapter 4.7), and the grid-interactive obligations that keep a gigawatt load online through a fault (Chapter 4.10). This chapter is about seeing and operating that machine — the metering that tells you where the power went, the power-quality instrumentation that tells you whether the waveform is healthy, the telemetry-driven control loops that make GPU load steps survivable, and the commissioning and day-2 electrical operations that keep the whole thing safe and inside spec. Observability is not a feature you bolt on; it is the precondition for every other decision in Part 4 actually holding in production.

Each fork in this chapter carries a downstream cost: how deep to meter, how fast to sample, open- versus closed-loop smoothing, converged versus federated DCIM/EPMS, and the commissioning level you hand over against. The recurring theme: AI loads broke the assumptions that classic data-center metering and power-quality practice were built on. A legacy hall drew a smooth, diversified, slowly-varying load that a monthly revenue meter and an annual power-quality audit described adequately. A training cluster is the opposite — a 100%-non-linear, sub-second-synchronized load that can swing tens of megawatts in milliseconds — and instrumentation built for the old world is blind to exactly the events that matter.

Metering in an AI factory is a hierarchy, and the first decision is how far down it reaches. At the top sits the revenue meter at the point of interconnection — utility-owned or customer-owned per Chapter 4.3, accuracy-class 0.2S or 0.5S, the meter the tariff is settled against and the only one the utility cares about. Below it, the question is how many layers of sub-metering you instrument: the medium-voltage feeders, the pod/block transformers, the LV switchboards and busway runs, the rack PDUs, and — at the deepest tier — branch-circuit and outlet-level metering inside the PDU (Chapter 4.6).

The fork is real because each layer costs money and bandwidth, and the payoff is attribution. Meter only at the POI and you know your bill but nothing else — you cannot tell a tenant what they owe, cannot attribute a PUE excursion to a pod, cannot see which busway is approaching its ampacity, and cannot reconcile the IT-reported power draw against what the facility actually delivered. The deepest tier — per-outlet metering at the PDU — is what makes a multi-tenant colo billable, what lets you enforce a rack-level power budget, and what feeds the energy-attribution that Chapter 15.1 needs to compute a defensible PUE/TUE. The downstream cost of under-metering is a permanent inability to answer questions the business will ask later, retrofittable only by opening live switchgear.

A second axis is what each meter measures. A cheap meter reports kWh and maybe RMS current. A power-quality-capable meter reports per-phase voltage and current, real/reactive/apparent power and power factor, individual harmonic magnitudes and THD, and — critically for AI — sag/swell/transient events with timestamps. The decision to specify PQ-capable meters at the right tiers (typically the MV feeders and the main LV boards) is what turns a metering hierarchy into a power-quality monitoring network without a second parallel instrumentation buildout.

AI halls are 100% non-linear loads. Every PSU, every UPS rectifier, every VRM draws current in pulses, not sinusoids, and the aggregate injects harmonic current back toward the source. IEEE 519-2022 is the recommended-practice that governs this, and its structure drives a specific monitoring decision: it sets limits at the point of common coupling (PCC) — the boundary between you and the utility, or between you and a shared tenant bus — not at every individual load. Voltage distortion at the PCC is capped (for 1–69 kV systems, 5% THD-V and 3% per individual harmonic; below 1 kV, 8% THD-V), and current distortion (TDD) is bounded on a sliding scale keyed to the short-circuit-to-load ratio I_sc/I_L — from 5% TDD on a weak bus to 20% on a stiff one. → harmonic mitigation hardware (K-rated transformers, active front ends, active filters) is the canonical subject of Chapter 4.4; here the decision is how you measure compliance.

And measurement is where the fork bites. IEEE 519 compliance is conventionally demonstrated as a snapshot — a power-quality engineer parks an analyzer at the PCC for a week, captures 10-minute aggregated statistics per IEC 61000-4-30 Class A, and produces a report. That satisfies the utility. It does not protect the GPUs. A synchronized training step is a millisecond-scale event; the voltage sag it induces on a stiff-but-not-infinite bus lives and dies inside a single cycle. A 10-minute statistical aggregate averages it into invisibility. If you want to see the events that actually throttle accelerators and trip undervoltage ride-through, you need continuous waveform capture with sub-cycle resolution and event triggering, not a compliance snapshot. The two are different instruments with a 5–10x cost delta, and conflating them is the most common power-quality blind spot in AI facilities.

The physics and mitigation of GPU power transients — the synchronized load steps, the chip-to-BBU-to-BESS absorption spine — are the canonical subject of Chapter 4.5. This chapter owns the observability and control half: the telemetry that feeds smoothing, and the decision of whether smoothing runs open- or closed-loop. That decision determines whether your facility is a passive victim of its own load steps or an actively-managed grid citizen.

Open-loop smoothing bakes the behavior in at commissioning: firmware ramp-rate limits, idle-time floors, and power caps set once and left. NVIDIA's GB300 NVL72 ships the building blocks for this — roughly 65 J/GPU of capacitive energy storage in the power shelf, a GPU-burn mechanism that bleeds energy on ramp-down, and ramp-rate controls that together cut peak grid demand by up to ~30%. Configured open-loop, those features run on fixed parameters regardless of grid state. Closed-loop smoothing instead drives the same knobs — power caps, ramp rates, idle floors — live, from telemetry: the GPU's own power and SoC reporting (NVIDIA SMI), the rack BBU and facility BESS state-of-charge, and the grid-stress signal from the EPMS. When the grid is stressed or the BESS is depleted, the loop tightens caps and slows ramps; when headroom returns, it relaxes them to recover goodput. The control surface is Redfish (the standardized out-of-band management API) and SMI; the inputs are the metering and PQ network this chapter is about.

The fork's downstream cost runs in both directions. Open-loop is simpler, has no live control-authority question, and cannot misbehave dynamically — but it leaves goodput on the table, because it must be tuned for the worst case and applies that conservatism always. Closed-loop recovers that goodput but introduces a genuine governance problem: when grid stress and a training deadline collide, something decides whether to throttle the GPUs or stress the grid, and that authority must be explicitly assigned — to the EPMS, to the workload scheduler, or to a negotiated handshake between them. Leaving it implicit is how a facility ends up either silently throttling revenue workloads or violating a grid-services commitment it forgot it had made. → the grid-side obligation that constrains this loop is Chapter 4.10; the goodput-vs-availability framing is Chapter 12.2; demand-response revenue that closed-loop control unlocks is Chapter 15.8.

Two platforms describe the electrical machine, and the decision is how they relate. The EPMS (Electrical Power Monitoring System) is the SCADA-grade, deterministic, often safety-rated system that watches the power chain in real time — breakers, relays (IEC 61850), meters, ATS/STS state, fault events — and is the system of record for protection and electrical events. DCIM (Data Center Infrastructure Management) is the broader operational layer: asset and capacity management, power/cooling/space, environmental sensors, increasingly a digital twin, and the home of cross-domain analytics and predictive maintenance. They overlap on power telemetry, and the seam between them is where most operational blind spots live.

The fork is converged versus federated. A converged platform — one vendor's DCIM ingesting the EPMS natively — gives a single pane and one data model, at the cost of lock-in and the risk that the safety-critical EPMS function is now entangled with a non-deterministic IT platform. Federated systems keep the EPMS as an independent, hardened, real-time system and define an explicit data contract by which it publishes to DCIM (commonly over a message bus or a historian, with the EPMS retaining autonomous protection authority). Federated is the more defensible posture for anyone who takes the view that protection must never depend on the analytics layer being up — but it only works if the data contract is real: defined tags, defined latencies, defined failure modes. An undocumented seam between EPMS and DCIM is the place where a meter reads one number, the IT stack reports another, and nobody can say which is right.

The hardest part of the integration is facility-to-IT, not facility-to-facility. The EPMS speaks Modbus, IEC 61850, and BACnet; the IT fleet speaks Redfish, IPMI, SNMP, DCGM/Prometheus, and PMBus down at the power shelf. SoC management across hundreds of rack BBUs is a concrete instance: the facility BESS state-of-charge lives in the EPMS, but each rack BBU's SoC is reported out-of-band via Redfish to the IT management plane. Closed-loop smoothing requires both views to be reconciled in one place at one cadence — which is precisely the converged-vs-federated decision made concrete. → the IT-side telemetry pipeline (DCGM, Prometheus) is built out in Chapter 14.2; the digital-twin/agentic-ops direction of DCIM is Chapter 14.7.

Commissioning is where all of the above is proven — or where the gap between design and reality is discovered, ideally before handover and not during a production training run. The industry runs a five-level (L1-L5) taxonomy: L1 factory acceptance testing of individual components, L2 site acceptance on delivery, L3 pre-functional checks, L4 functional performance testing of each system, and L5 integrated systems testing (IST) — the full, coordinated, failure-injection exercise where utility, generator, UPS, BESS, cooling, and IT load are run together through loss-of-power and load-step scenarios. The handover decision is which level you accept against, and for an AI factory the answer must be L5: the failure modes that matter are interactions, not component faults.

The defining commissioning fork for AI is the load profile you validate against. Traditional commissioning uses resistive load banks — a steady, linear, well-behaved load that proves the power chain can carry rated current and reject heat. That is necessary and insufficient. A resistive load bank cannot reproduce the two things that define an AI load: it is non-linear (it injects no harmonics, so it tests nothing about IEEE 519 behavior or the active front ends), and it is smooth (it executes no synchronized step, so it tests nothing about transient ride-through, BBU/BESS response, or the closed-loop smoothing logic). A cluster that passes a resistive load-bank test at rated power can still trip on the first 50-megawatt synchronized training step, because the step was never in the test plan.

Step-load validation closes that gap. The mature practice uses reactive and electronic load banks that can both inject harmonics (proving PQ behavior under realistic non-linearity) and execute programmed load steps that emulate the GPU ramp profile — validating that capacitance, BBUs, BESS, and the smoothing control loop actually catch the transient as designed, and that ride-through holds through the induced sag. The downstream cost of skipping step-load validation is the worst kind: the facility passes commissioning, the SLA clock starts, the first real workload lands, and the transient behavior that was never tested takes the cluster down — now a production incident with a customer attached, not a punch-list item.

The protection studies that produce a safe, selectively-coordinated electrical system — short-circuit/fault-duty analysis, time-current-curve coordination (IEEE 242), and arc-flash incident-energy calculation (IEEE 1584 / NFPA 70E) — are deliverables of the design phase, canonically Chapter 4.2. This chapter owns their operational half: keeping those studies live through every change, and running the day-2 electrical-safety program they imply. Selective coordination is a property that degrades every time a breaker setting is changed, a transformer is added, or a source configuration shifts. The operational discipline is a maintained coordination study and an as-operated single-line that the EPMS reflects in real time — so that a fault clears at the nearest device and takes down one pod, not the whole hall.

Arc-flash operations follow the same pattern: the IEEE 1584 incident-energy study yields the arc-flash boundary and PPE category for every piece of equipment, but those labels are only as good as the available fault current, which changes when sources change. The operational requirement is that the labels track the as-operated configuration and that energized work follows the NFPA 70E hierarchy of controls — which, for an AI factory at scale, increasingly means designing for de-energized maintenance (redundant paths that let you isolate and lock out a section without dropping load) rather than relying on PPE for live work.

The genuinely new operational hazard is DC touch-safety. The shift to ±400 V and 800 VDC distribution (Chapter 4.7) introduces a class of risk that AC-trained electrical operations teams have not internalized: DC arcs do not self-extinguish at a current zero the way AC arcs do, so DC arc-flash and sustained-arc behavior differ fundamentally; the systems are often ungrounded/IT-earthed, which means a first ground fault is silent and the second fault is the dangerous one (canonical DC grounding and isolation monitoring in Chapter 4.11); and solid-state DC breakers behave nothing like the electromechanical devices technicians know. The operational consequence is that the metering and monitoring network must surface DC isolation status and first-fault alarms as first-class operational signals, and the safety program must retrain for DC — because the most likely failure is not a design error but an experienced AC technician applying AC-correct instincts to a DC bus.