Facility Water Loops & Warm-Water Cooling

The CDU isolates the rack's technology-cooling loop from the building. Everything on the other side of that heat exchanger is the facility water system (FWS): the pipes in the pipe rack, the pumps in the plant room, and the water that carries the GPUs' heat to the sky. One decision sits upstream of almost every mechanical choice in it: how warm is the water you send to the CDUs? It is a design-basis input first and an operating knob second. Set against the site's climate, it decides how many hours a year a compressor runs, how the water-versus-energy trade falls, and whether the waste heat is a commodity or something you pay to dump.

That loop temperature organizes this chapter. We map the ASHRAE liquid classes (W17 through W+) and what each commits you to, then show why warmer water buys economizer hours and heat-reuse grade. The temperature then forces four pieces of engineering: loop topology and flow balancing, the delta-T discipline the plant depends on, water chemistry and treatment across a 10–15 year life, and fill, flush and commissioning. Heat rejection itself (chillers, towers, dry coolers, economizers) is Chapter 5.8; the economics of selling the heat are Chapter 15.5. Here we engineer the water that connects them.

ASHRAE TC 9.9 bands the facility water loop the way its air classes (A1–A4) band supply air. Each liquid class is named for the maximum allowable supply (entering) water temperature the equipment is rated to accept, in °C. The 5th edition added W40 in 2021 and renamed W1–W5 as W17, W27, W32, W45 and W+, extending the ladder for AI density. Every class shares a 2 °C lower bound. That is a freeze-protection floor, not a dew-point margin: the dew-point floor is set separately against measured space conditions (Chapter 5.12).

A class is a capability, not an operating point. It is the warmest water the plant must be able to deliver and the IT is rated to accept. Day-to-day supply sits inside the class and inside the rack's own TCS limit, with the remainder held as excursion and derate headroom. What the class buys in compressor hours, water and economizer hours depends on the site's dry- and wet-bulb bins and the rejection plant's approach, a calculation Chapter 5.8 runs. The class fixes only the upper bound.

The strategic content of the table is the diagonal. Walk warmer and you can take the compressor out of more operating hours and make the waste heat worth something, where the climate and the customer's heat sink support it. But the margin between your supply water and the chip's junction limit shrinks. The QCT GB200 NVL72 reference accepts up to 45 °C at the liquid inlet and 65 °C at the return. Those are two separate maxima, not an operating pair. Behind a CDU with a 3–5 K combined approach and distribution rise (an example, not a CDU rating), a 45 °C rack inlet needs facility water at 40–42 °C, at or just above W40's 40 °C limit. A 45 °C facility loop would already put that rack over its inlet maximum once the CDU approach is added. Warm water trades thermal margin for economizer hours. → The junction-to-coolant resistance budget that makes this tight is Chapter 5.1; the CDU approach that consumes part of the margin is Chapter 5.6.

Warm-water cooling can improve two things at once, economizer hours and heat-recovery lift, while spending the component temperature margin that pays for both. The physics is simple: heat flows downhill, so a loop warmer than the outside air can reject its heat through a dry cooler with no compressor. An 18 °C loop in air warmer than the water it must deliver has to push heat uphill some other way: evaporative assist where water use is permitted, a compressor, or a heat-load cut you agree to accept. The compressor adds energy, capex and a mechanical failure path. A 40–45 °C loop gives a dry cooler a larger temperature gap in the same air, and can shrink the compressor to hot-bin trim when the coil's capacity and approach allow it. The cooling load then moves toward pumps and fans. The annual weather bins and plant curves in Chapter 5.8 quantify how far.

The second prize is the heat itself. A chilled loop returning at ~25–30 °C is lukewarm beside a district network asking for 60–80 °C, so that customer needs a substantial heat-pump lift. A lower-temperature customer asks for less, and usefulness follows the customer's supply temperature and coincident demand. A loop supplied at 45 °C returns at ~55–65 °C if its load and flow produce a 10–20 K rise. That rise is an arithmetic example, not a W45 requirement. When the customer's supply and return temperatures and the exchanger approaches leave only a short lift, the waste heat stops being a cost and becomes a commodity. Select the heat pump at those conditions to get its coefficient of performance. Heat reuse is an engineering choice with a bill attached. Where Germany's reuse obligation applies, a colder loop can pay for more heat-pump lift on every kilowatt-hour delivered to the same customer; Chapter 15.5 has the dated legal thresholds and whom they cover. → The offtake economics and the chicken-and-egg problem of finding a heat buyer are Chapter 15.5; the engineering of the lift is Chapter 5.9.

The facility loop's job is to move a fixed quantity of heat from a few hundred CDUs to the heat-rejection plant while holding every CDU's supply temperature in a tight band, however the IT load shifts across the hall. Two topology forks define how, and each has a characteristic failure mode.

Primary/secondary (decoupled) vs variable-primary. The classic mission-critical pattern decouples a constant-flow production loop (pumps through the chillers or heat exchangers) from a variable-flow distribution loop (pumps to the CDUs). A low-loss header or decoupler sharing the same fluid inventory joins them, so the rejection plant sees stable flow while distribution follows load. It is robust and easy to stage, but it carries a second set of pumps and their parasitic energy. Variable-primary flow collapses the two into one variable loop with bypass control. That means fewer pumps, lower parasitic load and lower capex, at the cost of a harder control problem: the plant must protect minimum flow through the rejection equipment as the distribution side throttles. For a warm-water DLC plant whose premise is efficiency, the parasitic-pump saving of variable-primary is attractive. For a plant that must ride through synchronized GPU load slams without a chilled-water buffer's thermal inertia, the predictability of decoupled flow is worth the extra pumps. → The transient and ride-through side of this choice is Chapter 5.12.

Flow balancing across the hall. A loop feeding 200 CDUs is a hydraulic network in which the nearest CDU wants to hog flow and the farthest is starved. Reverse-return piping (Tichelmann) equalizes path length so every branch sees a similar pressure drop. Direct-return is cheaper but needs balancing valves and commissioning effort to keep the far racks from running hot. At the rack manifold the balance target is tight: every branch must stay between the minimum and maximum flow its equipment permits, because an out-of-balance branch shows up as a hot GPU, not just an inefficiency. → The in-rack manifold side is Chapter 5.4; worst-case-branch flow validation at commissioning is Chapter 13.5.

Every pipe diameter, pump size and heat-exchanger area in the facility loop is sized against one assumed number, the design temperature rise (delta-T) between supply and return. Physics fixes the relationship: for a given heat load, flow is inversely proportional to delta-T. Design for a wide delta-T (say 12 °C) and you move the heat with less water, so pumps and pipes are smaller and cheaper. Design for a narrow one and everything grows. On the rack side, OCP's guidance for PG25 cold-plate loops keeps the coolant rise between 7.5 and 12 °C at about 1.25–2.0 L/min per kW, with a nominal 10 °C and 1.5 L/min per kW. It is a technology-loop (TCS) figure, not a facility-loop target. Set the FWS delta-T and flow from your own heat load, approved fluid, pressure budget, control range, warranty and the CDU's four-port conditions.

The risk is less the design number than failing to achieve it. Low-delta-T syndrome is the chronic disease of large chilled and warm-water plants. Three-way-valve bleed-by, oversized coils, dirty heat exchangers or plain control sloppiness make the loop return colder than designed. The plant then pushes more flow to move the same heat, the pumps run harder and the rejection equipment loses efficiency. The symptom compounds, because higher flow shrinks the delta-T further at the same load. The result is a plant that hits its flow limit long before its thermal limit and strands cooling capacity you paid for. Design for an honestly achievable delta-T, use two-way valves and proper sequencing, and meter return temperature per branch, so the syndrome is caught as drift rather than discovered as a capacity wall. Before treating a low delta-T as a fault, check that it is excess bypass and not a genuine fall in load. → The metering that makes this observable is Chapter 4.12; the controls tuning that keeps valves from hunting is Chapter 5.12.

Required flow at 10.0 K is 459 L/min; at 5.00 K it is 919 L/min. At 600 L/min the reduced rise carries only (600/60) × 4.18 × 5.00 = 209 kW, leaving a 111 kW shortfall. Restore the useful ΔT or change the flow path before adding IT load.

The crossover at 600 L/min is 320/[(600/60) × 4.18] = 7.66 K. At half heat, 459 L/min and 5.00 K carry 160 kW and are consistent, so the issue becomes part-load pump energy rather than a capacity failure. 5.1 has the heat balance and 5.13 the pump/system curve; the ASHRAE CDU rating boundary keeps the CDU's internal heat separate.

The facility loop is a closed mixed-metal system, hydraulically isolated from the TCS cold plates by the CDU heat exchanger and circulating treated water or a water/glycol mix for the life of the building. Left untreated it does three things, all of which eventually cut heat-exchanger capacity and cooling reliability: it corrodes, it grows biology, and (if glycol is present) it oxidizes into acid. Treatment is not optional housekeeping. It stands between a clean loop and a slow loss of capacity.

Corrosion. A mixed-metal loop sets up galvanic couples. Set the inhibitor package and pH from the project's wetted-material matrix, the local water analysis, the water-treatment vendor and the CDU/OEM limits. Where aluminum is present, water outside the approved formulation's pH range can attack its protective film, so set the limit for your alloy and fluid. Rising dissolved iron can signal corrosion, but judge it against the baseline and action limit in your fluid schedule. Corrosion products are also the particulate that fouls facility-loop filters and the CDU heat exchanger. Biology. Warm water is a thermophile incubator, so a closed loop needs a documented biological-control plan, though a separate biocide is not automatically required. Biofilm insulates heat-transfer surfaces and harbors microbiologically influenced corrosion. (The Legionella exposure sits on the open-tower side of the plant, governed by ASHRAE 188, and is a Chapter 5.8 problem.) Glycol oxidation. Where freeze protection or biostasis calls for propylene glycol (commonly PG25), the glycol slowly oxidizes in the presence of oxygen, heat and metal catalysts into glycolic, lactic and formic acids. The acids can consume inhibitor reserve and turn the loop corrosive, so read pH and reserve together against the formulation's limits rather than declaring depletion from pH alone. How often TCS and FWS fluid must actually be replaced over 10–15 years is still being answered in the field, which is why monitoring matters more than any single additive.

Online pH and conductivity can expose a chemistry swing between grab samples, and a quarterly-only program can leave that swing unseen for months. Pair the sensors with supplier-specified laboratory checks for inhibitor reserve and dissolved metals, which the sensors do not measure. Set sampling locations, baselines, intervals and adverse-trend triggers, and give one owner the authority to act on comparable results. Skipping this is not abstract: a corroded, fouled loop strands cooling capacity in a building whose economic case is keeping GPUs at full clock.

The batch fails because 6.0 ppm is not below 5.0 ppm. Hold first fill. The fluid supplier investigates the water or formulation source and replaces or corrects the batch, and a conforming resample releases it. The pass line is strictly below 5.0 ppm, so a result of exactly 5.0 ppm also fails the adopted "less than" criterion. Agree the measurement uncertainty and the decision rule before releasing a real borderline result. Do not substitute the looser in-service chloride limit from OCP Table 4 for this fresh-fluid decision.

The fluid owner keeps the formulation and lot, the approved wetted list, sampling location, test method, units and acceptance criteria alongside the certificate and lab report. Establish the baseline at fill, and resample after incompatible make-up, repairs or adverse trends; the supplier sets the ongoing analytical interval and additive corrections. OCP PG guidance, Tables 1 and 4 and §6.3 distinguishes fresh and operating fluid. The fluid limits live in this chapter; 13.5 holds the release record and 14.5 runs the maintenance program.

The facility loop is a charged pressure system that must be clean at fill, leak-tight at pressure and balanced at flow before a single GPU is energized, and the order of operations is not negotiable. Piping design follows from the delta-T and flow decisions above. Size pipe to carry design flow without excessive pressure drop or erosion-corrosion, within the material's velocity limit. Choose wetted materials and electrical paths that limit galvanic attack, and reserve pipe-rack space and knockouts for the density ramp the building will see. The structural, surge and water-hammer, NPSH and code-basis (ASME B31.x / EN 13480 / PED) mechanics of charged pipe are deep enough for their own chapter, Chapter 5.13; the weight of charged pipe on the slab is Chapter 6.2.

Fill and flush is where AI-density loops differ most from legacy chilled water. A new piping system always contains mill scale, flux, weld slag and cutting debris. Debris left on the facility side clogs the FWS filters and the facility passages of the CDU exchanger. The TCS circuit needs a separate flush to the rack OEM's cleanliness spec before coolant reaches the cold-plate channels. The protocol is a staged clean and multi-pass flush: dislodge debris at a flow the isolated circuit can tolerate, apply chemical cleaning or passivation where the supplier requires it, and repeat compatible rinse passes until the chemistry and cleanliness endpoints pass. The supplier-approved sequence must also cover final fill and preservation, and you verify it by fluid quality (turbidity, conductivity, particle count), not by calendar. Run the hydrostatic test on the isolated pipe, filled and vented with a compatible test fluid. The approved plan decides where it falls relative to cleaning, drying or preservation, final treated fill and air removal (Chapter 5.13). Keep incompatible residue out of the final fluid and vent trapped air before it disrupts pump flow and heat transfer. At gigawatt scale the flush is itself a water event: a once-through commissioning flush can dominate year-one WUE even in a dry-cooled design. Closed-loop flush services that capture, treat and reuse the water are therefore worth specifying rather than assuming. Vertiv's PurgeRite NearZero, launched in June 2026, reported up to 78% less total water and up to 91% less haul-off in selected deployments (vendor-reported). Commissioning then proves the loop end to end: leak-tight at test pressure, every branch flowing within tolerance under worst-case-branch conditions, delta-T achieved at load, and leak-detection response time within acceptance. → Integrated cooling acceptance, flush acceptance criteria and the worst-case-branch flow test are Chapter 13.5; leak-detection and serviceability engineering is Chapter 5.11.

The recurring facility-loop mistakes all come from treating a design-basis number as adjustable later:

• Chilled by default, margin unpriced. A W17/W27 loop chosen because it feels safe adds compressor hours in any weather the economizers cannot cover, and the heat it rejects needs more lift to reach the same buyer. Water use follows the rejection plant you pick, not the loop. Choose the rack's TCS limit, the CDU approach and the FWS setpoint together, then price chilled against warm with the site's bins. A rack capped at a 45 °C coolant inlet can take facility water at 40 °C or below, which leaves room for the illustrative 3–5 K of CDU approach plus distribution rise.
• Designing a wide delta-T you can't hold. Sizing pumps and pipes for a 12 °C delta-T and then operating at 6 °C because of valve bleed-by and dirty coils. Low-delta-T syndrome doubles the flow, strands cooling capacity and surfaces as a wall of hot racks long after the plant is built.
• Skimping the flush. Treating fill and flush as a legacy chilled-water plumbing step rather than a cleanliness gate. Weld slag and mill scale left in the facility loop foul its filters and the CDU exchanger, and a dirty TCS flush puts the same debris into cold-plate channels, turning a commissioning shortcut into a chronic goodput tax.
• Treating warm water as free ride-through. Banking the chiller-less efficiency without checking whether the stored water still takes part in the actual upset, or whether its remaining temperature headroom can carry the heat. The first CDU trip then teaches that seconds, not minutes, is the new ride-through.

Choose the facility supply temperature and the separate TCS fluid specification together, then hold fill and operation inside the approved equipment and formulation limits. A hotter supply spends component margin; an unapproved refill spends reliability and can void support before the plant saves any energy.