The Definitive Guide toAI Data Centers

17 Parts · 173 chapters

0Foundations & How to Use This Guide

0.1Orientation: The AI Data Center as a Single Co-Designed Machine 0.2How to Read This Guide: Decisions, Consequences & Reference Data 0.3Vocabulary, Mental Models & the Metric Stack 0.4The Standards & Specifications Landscape (Living Index)0.5Reliability, Redundancy & Availability: The Design-Basis Primer

1Strategy, Workload Archetypes & Economics

1.1The Archetype Decision Framework: Workload Is the Master Variable 1.2Training Data Centers: Synchronous, Dense, Checkpointable 1.3Inference Data Centers: Bursty, Distributed, Always-On 1.4Post-Training, Fine-Tuning & RL: The Hybrid Middle 1.5Edge Inference & Distributed Micro-Datacenters 1.6Procurement Archetypes: Build vs Buy vs Rent 1.7The Requirements-and-Consequences Matrix 1.8Business Models, Economics & ROI

2Project Delivery, Schedule, Procurement, Contracts & Risk

2.1Program & Project Management: The Integrated Master Schedule & Critical Path 2.2Delivery Models & the Owner's Organization 2.3Long-Lead Procurement & the End-to-End Equipment Supply Chain 2.4The Contract Stack & Commercial/Legal Framework 2.5Project Finance & Capital Formation (Mechanics)2.6Insurance & Risk Transfer 2.7Simulation-Driven Design & the Digital Twin as a Design-Validation Tool

3Site Selection, Power Procurement & Permitting

3.1Site Selection Strategy & the Reordered Criteria Hierarchy 3.2Grid Interconnection, Queues & Speed-to-Power 3.3Power Availability & Power-Cost Structure 3.4Energy Supply Strategy: Grid PPA, BYOP & Co-Location 3.5On-Site & Bring-Your-Own-Power Generation (Energy-Supply Strategy)3.6Fiber, Latency & Network Connectivity (Secondary Screen)3.7Water Availability, Sourcing & Climate-Driven Cooling Strategy (Siting Gate)3.8Land, Geotechnical, Seismic & Flood Diligence (Secondary Screen)3.9Permitting, Regulatory, Environmental & the Critical Path 3.10Tax Incentives, Fiscal Structuring & Economic Development 3.11Community Relations, Opposition & Social License 3.12Geopolitics, Sovereignty, Export Controls & Data Residency 3.13Market Clusters & the Site-Scoring Playbook

4Electrical & Energy Infrastructure

4.1Power Topology Foundations & Voltage Selection 4.2Utility Interconnect, On-Site Substation & MV Distribution 4.3Substation & Transmission Ownership, Operations & NERC Compliance 4.4Transformers, Harmonics & the AI Non-Linear-Load Problem 4.5UPS & Energy Storage: From Ride-Through to Transient Absorption 4.6LV Distribution: Busway, PDUs, RPPs & Rack Power 4.7The DC Power Revolution: 48V → ±400V → 800V & Disaggregated Sidecar Power 4.8On-Site Generation: Electrical Integration 4.9Fuel-Supply & Gas-Process Engineering 4.10Grid-Interactive Behavior: Ride-Through, Reactive/Voltage Support & Frequency Response Toward the POI 4.11Grounding, Bonding, Earthing, Lightning Protection, SPD & EMC 4.12Metering, Power Quality, Monitoring & Electrical Operations

5Cooling & Thermal Management

5.1Thermal Fundamentals & the Density Wall 5.2Air Cooling at the Limit 5.3Rear-Door Heat Exchangers & Air-Assisted Liquid Cooling (The Bridge)5.4Direct-to-Chip Liquid Cooling (DLC) — The 2026 Default 5.5Immersion Cooling (Single-Phase & Two-Phase)5.6CDUs & the Secondary Loop 5.7Facility Water Loops & Warm-Water Cooling 5.8Heat Rejection: Chillers, Dry Coolers, Towers, Adiabatic & Economizers 5.9Heat Reuse & Waste-Heat Recovery (Engineering)5.10Retrofitting Air-Cooled Facilities for Liquid 5.11Thermal Design, Reliability, Leak Detection & Commissioning 5.12Cooling-Controls Transient Dynamics & Setpoint Stability 5.13Facility Piping & Pressure-System Mechanical Engineering

6The Building: Civil, Structural, Fire/Life-Safety & Construction Execution

6.1Building Typologies & Data-Hall Layout 6.2Structural & Civil Engineering for Dense Liquid-Cooled Halls 6.3Building Envelope, Architecture & Site Civil Works 6.4Modular & Prefabricated Construction 6.5Fire Detection, Suppression & Life-Safety 6.6Construction Execution, Sequencing & Phased Turnover 6.7Rack Civil Integration: Mass, Floor-Loading & Seismic Anchoring 6.8Acoustic & Emissions Engineering Design 6.9Environment, Health & Safety (EHS) Across Build & Operate

7Compute, Silicon & System Integration

7.1Accelerator Landscape & Taxonomy 7.2NVIDIA Accelerators: Hopper → Blackwell → Vera Rubin → Rubin Ultra → Feynman 7.3AMD Instinct & the Open Challenger 7.4Hyperscaler XPUs: TPU, Trainium/Inferentia, Maia, MTIA 7.5Custom ASICs & the Merchant-Silicon Disruption 7.6HBM: The Binding Constraint on AI Compute 7.7Advanced Packaging & the Integration Substrate 7.8Host CPUs, GPU:CPU Ratios & System Composition 7.9Software Ecosystems & Lock-In 7.10Precision, Quantization & the Compute-Memory Tradeoff 7.11Accelerator Selection, TCO & Procurement Strategy 7.12On-Package Power Delivery & Power Integrity 7.13The Rack as Integration Unit 7.14Server & System Integration 7.15Deployment Velocity & Cabling at Scale

8Networking, Fabrics & Optics

8.1Network Fundamentals & AI Traffic Characterization 8.2Scale-Up Fabric (Intra-Node / Intra-Rack)8.3Network Silicon: Switch ASICs, NICs & DPUs 8.4Scale-Out Fabric: Protocols, Standards & Transport 8.5Scale-Out Topology, Sizing & Oversubscription 8.6Congestion Control, Load Balancing & In-Network Compute 8.7Management, Out-of-Band Fabric & PTP/IEEE-1588 Timing 8.8Scale-Across: Multi-Campus & Cross-Region Fabric (DCI for Distributed Training)8.9Physical-Layer & Interconnect Taxonomy 8.10CPO, Fiber Plant & Structured Cabling

9Storage & Data

9.1Storage in the AI Lifecycle: Why It Determines GPU Efficiency 9.2Parallel & Distributed File Systems 9.3NVMe Tiers, GPUDirect Storage & the CPU-Bypass Data Path 9.4Checkpointing for Large-Scale Training 9.5Data Ingestion, Preprocessing & the Data-Loader Path 9.6Object Storage, Data Lakes & the Capacity Tier 9.7Inference & KV-Cache Storage: The New Memory Hierarchy 9.8Sizing, Data Gravity & Resilience 9.9The Data-Prep Supercomputer: Offline Data Processing

10Software, Orchestration & Service Delivery

10.1Orchestration Architecture & the Scheduling Plane 10.2Topology-Aware & Rack-Scale Scheduling 10.3Multi-Tenancy, Isolation & Resource Sharing 10.4Node Software Stack: Drivers, CUDA/ROCm, NCCL & Firmware 10.5Provisioning, Bring-Up & Infrastructure as Code 10.6Observability, Telemetry & GPU Health 10.7Fleet Reliability, Fault Tolerance & Autonomous Recovery 10.8MLOps & Training Frameworks 10.9Customer Onboarding, Delivery & Productization 10.10Data Governance, Privacy & the Training-Data Legal Regime 10.11Inference Serving Engineering: SLOs, Batching, Disaggregation & Goodput-Optimal Scheduling

11Security

11.1Threat Model, Assets & Security Levels for AI Infrastructure 11.2Physical Security: Siting, Zones & Kinetic/Drone Threats 11.3Supply-Chain Security & Hardware Provenance 11.4Hardware Root of Trust, Firmware & BMC Security 11.5GPU Confidential Computing & Trusted Execution 11.6Multi-Tenant & Workload Isolation Security 11.7Network Segmentation, Microsegmentation & Zero Trust 11.8Model & Weight Protection (At-Rest, In-Transit, In-Use)11.9Insider Threat & Human-Layer Security 11.10Cyber-Physical & Destructive Attacks on OT/Facility Systems 11.11Compliance, Certification & Governance 11.12Security Operations, Detection & Incident Response

12Reliability, Resilience & Standards

12.1Resilience Standards, Redundancy Topologies & Fault-Domain Engineering 12.2The AI-Cluster Reliability Rethink: Goodput vs Facility Availability 12.3Disaster Recovery, Business Continuity & Geographic Failover 12.4SLAs, Goodput Contracts & Availability Commitments 12.5Quantitative Reliability & Availability Modeling (RBD / FTA / Monte-Carlo)

13Commissioning & Go-Live

13.1Commissioning Fundamentals, Levels & Program Governance 13.2Documentation, Scripts & Acceptance Test Plans 13.3Electrical Power Acceptance (L3/L4)13.4Commissioning On-Site Generation & Microgrid Controls 13.5Cooling Acceptance: Air, Liquid-to-Chip & CDU Commissioning 13.6Level 5 Integrated Systems Testing (IST) & Failure-Mode Demonstration 13.7Network Fabric Commissioning & Validation 13.8GPU Node Burn-In, Diagnostics & Stress Validation 13.9Cluster-Scale Benchmarking, Reference Training & Storage/Scheduler Validation 13.10Staged Power/Load Ramp, Go-Live & Handover to Operations

14Day-2 Operations, Upgrades & Lifecycle

14.1Operational KPIs, Goodput & the Reliability Economics of AI Factories 14.2DCIM, Telemetry & Observability for GPU-Dense, Liquid-Cooled Facilities 14.3Component Failure Modes, Failure Rates & Fleet Reliability Data 14.4Reliability Engineering for Training (Operational)14.5Predictive & Preventive Maintenance of Power and Cooling Plant 14.6Spares Strategy, RMA Logistics & Repair Operations 14.7Capacity, Power & Thermal Management in Operation 14.8Firmware & Software Lifecycle Management at Fleet Scale 14.9Hardware Refresh, Depreciation Strategy, Decommissioning & ITAD 14.10Facility Decommissioning, Repowering & Site Remediation 14.11Operations Organization, Workforce, Talent & Incident Command 14.12Operational Procedures, Change Management & Human-Error Control 14.13Agentic Ops, RL Control & the Autonomy Ladder 14.14Continuous & Re-Commissioning on a Live Campus

15Sustainability & Efficiency

15.1Efficiency Metrics: PUE, WUE, ERF, REF & the Post-PUE Metric Stack 15.2Energy Efficiency: Cooling, Free Cooling, Setpoints & Power-Chain Losses 15.3Carbon, Clean Power Procurement & 24/7 Carbon-Free Energy 15.4Water Stewardship 15.5Heat Reuse & District Heating (Sustainability & Economics)15.6Embodied Carbon & Circularity Across the Lifecycle 15.7Regulation, Reporting & Disclosure Frameworks 15.8Grid Impact, Energy-Systems Integration & Grid Services

16Trends, Roadmaps & the Future

16.1The Power-Bound Era: Why the Bottleneck Moved to the Substation 16.2Subsystem Roadmaps 2026 → 2030 (Consolidated)16.3Software, Orchestration & Efficiency at the Frontier 16.4The Economics of the Build-Out 16.5Scenarios for 2030

§Appendices & Reference Data

AStandards & Specifications Cross-Reference Matrix BReference Designs & Worked Examples CDecision Tables & Calculators DNumbers Provenance & Forecast Register EGlossary, Phase-Gate Timeline & Learning/Community Map FFailure-Mode / FMEA Catalog GRegional & International Design Deltas: Consolidated Quick-Reference Crosswalk

Guide › Glossary › Tokens-per-joule

Tokens-per-joule

Inference energy efficiency measured as tokens generated per joule, a cross-vendor comparator that survives generations.

The Definitive Guide to AI Data Centers · 173 chapters · 2026Ask the Guide · Calculators · Register · Glossary · Feedback