Facility Water Loops & Warm-Water Cooling

The CDU isolates the technology-cooling loop at the rack from the building. Everything on the other side of that heat exchanger — the pipes in the pipe rack, the pumps in the plant room, the water that ultimately carries the GPUs' heat to the sky — is the facility water system (FWS), and it is governed by one decision that sits upstream of almost every mechanical choice in the building: how warm is the water you push to the CDUs? That number is a design-basis input first and an operating knob second: set against the site's climate, it decides how many hours a year a compressor runs, how the water-versus-energy trade falls, and whether the waste heat is a commodity or a thermodynamic nuisance you pay to dump.

That setpoint — the loop temperature — organizes this chapter. We map the ASHRAE liquid classes (W17 through W45+) and what each commits you to; trace why warmer water buys economizer hours and heat-reuse grade; and then work the engineering the temperature choice forces downstream: loop topology and flow balancing, the delta-T discipline the whole plant lives or dies on, water chemistry and treatment over a 10–15 year life, and the fill/flush/commissioning sequence. Heat rejection itself — the chillers, towers, dry coolers, and economizers the loop couples to — is Chapter 5.8; the economics of selling the heat are Chapter 15.5. Here we engineer the water that connects them.

ASHRAE TC 9.9's liquid-cooling guidance bands the facility water loop the way its air classes (A1–A4) band the supply air: each class is named for the maximum allowable supply (entering) water temperature the equipment is rated to accept. The 5th-edition refresh renamed and extended the ladder to keep pace with AI density, adding the warm classes the industry is now standardizing on. The numbers are the upper supply limit in °C; every class shares a 2 °C lower bound — a freeze-protection floor, not a dew-point margin; the dew-point floor is set separately against measured space conditions (Chapter 5.12).

The class is a capability, not an operating point: it is the warmest water the plant must be able to deliver and the IT is qualified to accept, and day-to-day setpoints sit below it (typically ~25–35 °C), holding the remainder as excursion and derate headroom. What the class buys in compressor, water and economizer hours falls out of the site's dry- and wet-bulb bins and the rejection plant's approach — the calculation Chapter 5.8 runs; the class fixes only its upper bound.

The strategic content of the table is the diagonal: walk warmer and you can move the compressor out of more operating hours and make the waste heat worth something, where the actual climate and customer heat sink support that exchange — but the thermal margin between your supply water and the chip's junction limit shrinks. The QCT GB200 NVL72 reference accepts up to 45 °C at the liquid inlet and 65 °C at the return — two separate maxima, not an operating pair — so a W45 loop run at its class limit has spent the rack's entire inlet margin before the CDU approach and the worst branch take their share. That is the warm-water bargain: thermal margin traded for economizer hours. → the junction-to-coolant resistance budget that makes this tight is Chapter 5.1; the CDU approach that consumes part of the margin is Chapter 5.6.

Warm-water cooling can improve two prizes at once — more economizer hours and less heat-recovery lift — while spending the component temperature margin that pays for both. The mechanism is simple thermodynamics: heat flows downhill, so a loop warmer than the outside air rejects its heat through a dry cooler with no compressor at all. An 18 °C loop in air warmer than its required leaving-water temperature needs another way to push heat uphill: permitted evaporative assistance, a compressor, or accepted heat reduction. That compressor adds energy, capex and a mechanical failure path. A 40–45 °C loop gives a dry cooler a larger temperature gap in the same air and can reduce the compressor to hot-bin trim when the coil’s capacity and approach close. The cooling load then moves toward pumps and fans; the annual weather bins and plant curves in Chapter 5.8 quantify how far.

The second prize is the heat itself. A chilled loop returning at ~25–30 °C is lukewarm beside a district network asking for 60–80 °C: that customer needs a substantial heat-pump lift. A lower-temperature customer asks less; usefulness follows its supply temperature and coincident demand. A warm loop operated at 45 °C supply returns at ~55–65 °C if its actual heat load and flow produce a 10–20 K rise, an arithmetic operating example rather than a W45 requirement — close enough to turn waste heat from cost to commodity where the customer’s supply, return and exchanger approaches leave a short heat-pump lift; select the pump at those conditions to establish its coefficient of performance. This is why heat reuse becomes an engineering choice with a bill attached: Germany’s applicable obligation can make a colder loop pay for more heat-pump lift on every kilowatt-hour delivered to the same customer. Chapter 15.5 owns the dated legal thresholds and applicability. → the offtake economics and the chicken-and-egg of finding a heat buyer are Chapter 15.5; the engineering of the lift is Chapter 5.9.

The facility loop's job is to move a fixed quantity of heat from a few hundred CDUs to the heat-rejection plant while holding every CDU's supply temperature within a tight band, regardless of how the IT load shifts across the hall. Two topology forks define how you do it, and each has a characteristic failure mode.

Primary/secondary (decoupled) vs variable-primary. The classic mission-critical pattern decouples a constant-flow production loop (pumps through the chillers/heat exchangers) from a variable-flow distribution loop (pumps to the CDUs) via a low-loss header or decoupler sharing the same fluid inventory, so the rejection plant sees stable flow while the distribution side modulates to load. It is robust and easy to stage, but it carries a second set of pumps and their parasitic energy. Variable-primary-flow collapses the two into one variable loop with bypass control — fewer pumps, lower parasitic load, lower capex — at the cost of a more demanding control problem: the plant must protect minimum flow through the rejection equipment as the distribution side throttles. For a warm-water DLC plant whose whole premise is energy efficiency, the parasitic-pump saving of variable-primary is attractive; for a plant that must ride through synchronized GPU load slams without the thermal inertia of a chilled-water buffer, the predictability of decoupled flow is worth the pumps. → the transient/ride-through dimension of this choice is Chapter 5.12.

Flow balancing across the hall. A loop feeding 200 CDUs is a hydraulic network in which the nearest CDU wants to hog flow and the farthest is starved. Reverse-return piping (Tichelmann) equalizes path length so every branch sees similar pressure drop; direct-return is cheaper but needs balancing valves and commissioning effort to avoid the far racks running hot. The branch-balance target at the rack manifold is tight — the minimum and maximum flow permitted by the named branch equipment — because an out-of-balance branch shows up as a hot GPU, not just an inefficiency. → the in-rack manifold side of this is Chapter 5.4; the worst-case-branch flow validation at commissioning is Chapter 13.5.

Every pipe diameter, pump size, and heat-exchanger area in the facility loop is sized against one assumed number: the design temperature rise (delta-T) between supply and return. The relationship is fixed by physics — for a given heat load, flow is inversely proportional to delta-T. Design for a wide delta-T (say 12 °C) and you move the heat with less water, so pumps and pipes are smaller and cheaper; design for a narrow delta-T and everything grows. One Dober PG25 planning illustration uses roughly 7.5–12 °C and ~1.25–2.0 L/min per kW on the secondary side. It is not a universal target; select the project ΔT and flow from the named equipment, approved fluid, heat load, pressure budget, control range and warranty.

The fork is not the design number but what happens when you fail to achieve it. Low-delta-T syndrome is the chronic disease of large chilled/warm-water plants: through three-way-valve bleed-by, oversized coils, dirty heat exchangers, or simple control sloppiness, the loop returns colder than designed, so the plant pushes more flow to move the same heat, so the pumps run harder and the rejection equipment loses efficiency — and the symptom compounds because higher flow further shrinks the delta-T at the same heat load; distinguish excess bypass from a legitimate fall in load before treating the low delta-T as a fault. The consequence is a plant that hits its flow limit long before its thermal limit and strands cooling capacity you paid for. The discipline is to design for an honestly-achievable delta-T, use two-way valves and proper sequencing, and meter return temperature per branch so the syndrome is caught as drift rather than discovered as a capacity wall. → the metering that makes this observable is Chapter 4.12; the controls tuning that prevents the valves from hunting is Chapter 5.12.

Required flow at 10.0 K is 459 L/min; at 5.00 K it is 919 L/min. At 600 L/min the reduced rise carries only (600/60) × 4.18 × 5.00 = 209 kW, leaving a 111 kW shortfall. Restore useful ΔT or change the qualified flow path before adding IT load.

The crossover at 600 L/min is 320/[(600/60) × 4.18] = 7.66 K. At half heat, however, 459 L/min and 5.00 K carry 160 kW and are consistent; the issue becomes part-load pump energy rather than a capacity failure. Use 5.1 for the balance and 5.13 for the pump/system curve, with the ASHRAE CDU rating boundary keeping internal heat separate.

The facility loop is a closed mixed-metal system, hydraulically isolated from TCS cold plates by the CDU heat exchanger, circulating treated water or a water/glycol mix for the life of the building. Left untreated it does three things, all of which eventually reduce heat-exchanger capacity and cooling reliability: it corrodes, it grows biology, and (if glycol is present) it oxidizes into acid. The treatment regime is not optional housekeeping; it is the thing standing between a clean loop and a slow-motion capacity loss.

Corrosion. A mixed-metal loop sets up galvanic couples; set the inhibitor package and pH from the project-specific wetted-material matrix, local water analysis, water-treatment vendor, and CDU/OEM limits. Where aluminum is present, water outside the approved formulation’s pH envelope can attack its protective film; set the limit for the named alloy and fluid. Rising dissolved iron can signal corrosion, but judge it against that fluid schedule’s baseline and action limit — corrosion products are also the particulate that fouls facility-loop filters and the CDU heat exchanger. Biology. Warm water is a thermophile incubator; a closed loop needs a qualified biological-control plan; a separate biocide is not automatically required, because biofilm both insulates heat-transfer surfaces and harbors microbiologically-influenced corrosion. (The Legionella exposure lives on the open-tower side of the plant, governed by ASHRAE 188 — that is a Chapter 5.8 problem.) Glycol oxidation. Where freeze protection or biostasis mandates propylene glycol (commonly PG25), the glycol slowly oxidizes in the presence of oxygen, heat, and metal catalysts into glycolic, lactic, and formic acids; acid formation can consume inhibitor reserve and turn the loop corrosive; interpret pH and reserve together against the formulation’s limits instead of declaring depletion from pH alone. The 10–15 year question — how often must the TCS/FWS fluid actually be replaced — is still being answered in the field, which is exactly why monitoring matters more than any single additive.

Online pH and conductivity can expose a chemistry swing between grab samples; a quarterly-only program can leave that swing unseen for months. Pair the sensors with supplier-specified laboratory checks for inhibitor reserve and dissolved metals, which those sensors do not measure. Set sampling locations, baselines, intervals and adverse-trend triggers with one owner authorized to act on comparable results. The consequence of skipping it is not abstract: a corroded, fouled loop strands cooling capacity inside a building whose entire economic case is keeping GPUs at full clock.

The batch fails because 6.0 ppm is not below 5.0 ppm. Hold first fill, have the fluid supplier investigate the water/formulation source and supply or correct the batch, then obtain a conforming resample. The crossover is chloride strictly below 5.0 ppm; an exact 5.0 ppm result also fails the adopted “less than” criterion. Measurement uncertainty and the decision rule must be agreed before a real borderline result can be released. Do not substitute the looser in-service chloride limit from OCP Table 4 for this fresh-fluid decision.

The fluid owner retains the formulation/lot, approved wetted list, sampling location, test method, units and acceptance criteria alongside the certificate and lab report. Establish the baseline at fill and resample after incompatible make-up, repairs or adverse trends; the supplier sets the ongoing analytical interval and additive corrections. OCP PG guidance, Tables 1 and 4 and §6.3 distinguishes fresh and operating fluid. This chapter owns the fluid envelope; 13.5 receives its release record and 14.5 executes the maintenance program.

The facility loop is a charged pressure system that must be clean at fill, leak-tight at pressure, and balanced at flow before a single GPU is energized — and the order of operations is not negotiable. Piping design follows from the delta-T and flow decisions above: pipe is sized to move design flow without excessive pressure drop or erosion-corrosion within the qualified material’s velocity envelope, and the wetted materials and electrical paths are chosen to limit galvanic attack, and the routing reserves pipe-rack space and knockouts for the density ramp the building will see. The structural, surge/water-hammer, NPSH, and code-basis (ASME B31.x / EN 13480 / PED) mechanics of the charged pipe are deep enough to be their own chapter — Chapter 5.13 — and the weight of charged pipe on the slab is Chapter 6.2.

Fill and flush is where AI-density loops differ most from legacy chilled water. A loop that feeds sub-millimeter cold-plate channels cannot tolerate the mill scale, flux, weld slag, and cutting debris that a new piping system always contains; left in, that debris fouls the CDU filters and the microchannels within weeks. The protocol is a staged clean and multi-pass flush: dislodge debris at a flow the isolated circuit can tolerate, apply chemical cleaning or passivation where the supplier requires it, and repeat compatible rinse passes until the chemistry and cleanliness endpoints pass. The supplier-approved sequence must also cover final fill and preservation — verified by fluid quality (turbidity, conductivity, particle count) rather than by calendar. The hydrostatic test uses the isolated pipe scope filled and vented with compatible test fluid; the approved plan sets its order against cleaning, drying or preservation, final treated fill and air removal (Chapter 5.13). Keep incompatible residue out of the final fluid and vent trapped air before it disrupts pump flow and heat transfer. At gigawatt scale the flush itself is a water event — a once-through commissioning flush can dominate year-one WUE even in a dry-cooled design — which is why closed-loop flush services that capture, treat, and reuse the water (e.g. Vertiv's PurgeRite NearZero, launched Jun 2026, vendor-claimed up to 95% less flush water) are becoming the thing to specify rather than assume. Commissioning then proves the loop end-to-end: leak-tight at test pressure, every branch flowing within tolerance under worst-case-branch conditions, delta-T achieved at load, and leak-detection response time within acceptance. → integrated cooling acceptance, the flush acceptance criteria, and the worst-case-branch flow test are Chapter 13.5; the leak-detection and serviceability engineering is Chapter 5.11.

The recurring facility-loop mistakes all come from treating one of its design-basis numbers as adjustable later:

• Chilled-by-default for margin you didn't price. Specifying a W17/W27 loop because the chip datasheet allows it and it feels safe — and silently committing the building to a compressor in the critical path, an evaporative tower's water bill, and waste heat too cold to sell. If the silicon accepts W45, going chilled is buying thermal insurance at the price of the plant's whole efficiency case.
• Designing a wide delta-T you can't hold. Sizing pumps and pipes for a 12 °C delta-T and then operating at 6 °C because of valve bleed-by and dirty coils — low-delta-T syndrome that doubles the flow, strands cooling capacity, and surfaces as a hot-rack wall long after the plant is built.
• Skimping the flush. Treating the fill/flush as a legacy chilled-water plumbing step rather than a sub-millimeter-channel cleanliness gate — and seeding the loop with weld slag and mill scale that fouls the CDU filters and cold plates within weeks of go-live, converting a commissioning shortcut into a chronic goodput tax.
• Treating warm water as free ride-through. Banking the chiller-less efficiency while forgetting that you never established whether the stored water still participates in the actual upset, or whether its remaining temperature headroom can ride through it — and discovering at the first CDU trip that seconds, not minutes, is the new ride-through.

Choose the facility supply temperature and the separate TCS fluid specification together, then hold fill and operation inside the approved equipment and formulation envelope. A hotter supply spends component margin; an unqualified refill spends reliability and can invalidate support before the plant saves any energy.