The Definitive Guide to
AI Data
Centers
Ask the Guide
17 Parts · 173 chapters
0
Foundations & How to Use This Guide
0.1
Orientation: The AI Data Center as a Single Co-Designed Machine
0.2
How to Read This Guide: Decisions, Consequences & Reference Data
0.3
Vocabulary, Mental Models & the Metric Stack
0.4
The Standards & Specifications Landscape (Living Index)
0.5
Reliability, Redundancy & Availability: The Design-Basis Primer
1
Strategy, Workload Archetypes & Economics
1.1
The Archetype Decision Framework: Workload Is the Master Variable
1.2
Training Data Centers: Synchronous, Dense, Checkpointable
1.3
Inference Data Centers: Bursty, Distributed, Always-On
1.4
Post-Training, Fine-Tuning & RL: The Hybrid Middle
1.5
Edge Inference & Distributed Micro-Datacenters
1.6
Procurement Archetypes: Build vs Buy vs Rent
1.7
The Requirements-and-Consequences Matrix
1.8
Business Models, Economics & ROI
2
Project Delivery, Schedule, Procurement, Contracts & Risk
2.1
Program & Project Management: The Integrated Master Schedule & Critical Path
2.2
Delivery Models & the Owner's Organization
2.3
Long-Lead Procurement & the End-to-End Equipment Supply Chain
2.4
The Contract Stack & Commercial/Legal Framework
2.5
Project Finance & Capital Formation (Mechanics)
2.6
Insurance & Risk Transfer
2.7
Simulation-Driven Design & the Digital Twin as a Design-Validation Tool
3
Site Selection, Power Procurement & Permitting
3.1
Site Selection Strategy & the Reordered Criteria Hierarchy
3.2
Grid Interconnection, Queues & Speed-to-Power
3.3
Power Availability & Power-Cost Structure
3.4
Energy Supply Strategy: Grid PPA, BYOP & Co-Location
3.5
On-Site & Bring-Your-Own-Power Generation (Energy-Supply Strategy)
3.6
Fiber, Latency & Network Connectivity (Secondary Screen)
3.7
Water Availability, Sourcing & Climate-Driven Cooling Strategy (Siting Gate)
3.8
Land, Geotechnical, Seismic & Flood Diligence (Secondary Screen)
3.9
Permitting, Regulatory, Environmental & the Critical Path
3.10
Tax Incentives, Fiscal Structuring & Economic Development
3.11
Community Relations, Opposition & Social License
3.12
Geopolitics, Sovereignty, Export Controls & Data Residency
3.13
Market Clusters & the Site-Scoring Playbook
4
Electrical & Energy Infrastructure
4.1
Power Topology Foundations & Voltage Selection
4.2
Utility Interconnect, On-Site Substation & MV Distribution
4.3
Substation & Transmission Ownership, Operations & NERC Compliance
4.4
Transformers, Harmonics & the AI Non-Linear-Load Problem
4.5
UPS & Energy Storage: From Ride-Through to Transient Absorption
4.6
LV Distribution: Busway, PDUs, RPPs & Rack Power
4.7
The DC Power Revolution: 48V → ±400V → 800V & Disaggregated Sidecar Power
4.8
On-Site Generation: Electrical Integration
4.9
Fuel-Supply & Gas-Process Engineering
4.10
Grid-Interactive Behavior: Ride-Through, Reactive/Voltage Support & Frequency Response Toward the POI
4.11
Grounding, Bonding, Earthing, Lightning Protection, SPD & EMC
4.12
Metering, Power Quality, Monitoring & Electrical Operations
5
Cooling & Thermal Management
5.1
Thermal Fundamentals & the Density Wall
5.2
Air Cooling at the Limit
5.3
Rear-Door Heat Exchangers & Air-Assisted Liquid Cooling (The Bridge)
5.4
Direct-to-Chip Liquid Cooling (DLC) — The 2026 Default
5.5
Immersion Cooling (Single-Phase & Two-Phase)
5.6
CDUs & the Secondary Loop
5.7
Facility Water Loops & Warm-Water Cooling
5.8
Heat Rejection: Chillers, Dry Coolers, Towers, Adiabatic & Economizers
5.9
Heat Reuse & Waste-Heat Recovery (Engineering)
5.10
Retrofitting Air-Cooled Facilities for Liquid
5.11
Thermal Design, Reliability, Leak Detection & Commissioning
5.12
Cooling-Controls Transient Dynamics & Setpoint Stability
5.13
Facility Piping & Pressure-System Mechanical Engineering
6
The Building: Civil, Structural, Fire/Life-Safety & Construction Execution
6.1
Building Typologies & Data-Hall Layout
6.2
Structural & Civil Engineering for Dense Liquid-Cooled Halls
6.3
Building Envelope, Architecture & Site Civil Works
6.4
Modular & Prefabricated Construction
6.5
Fire Detection, Suppression & Life-Safety
6.6
Construction Execution, Sequencing & Phased Turnover
6.7
Rack Civil Integration: Mass, Floor-Loading & Seismic Anchoring
6.8
Acoustic & Emissions Engineering Design
6.9
Environment, Health & Safety (EHS) Across Build & Operate
7
Compute, Silicon & System Integration
7.1
Accelerator Landscape & Taxonomy
7.2
NVIDIA Accelerators: Hopper → Blackwell → Vera Rubin → Rubin Ultra → Feynman
7.3
AMD Instinct & the Open Challenger
7.4
Hyperscaler XPUs: TPU, Trainium/Inferentia, Maia, MTIA
7.5
Custom ASICs & the Merchant-Silicon Disruption
7.6
HBM: The Binding Constraint on AI Compute
7.7
Advanced Packaging & the Integration Substrate
7.8
Host CPUs, GPU:CPU Ratios & System Composition
7.9
Software Ecosystems & Lock-In
7.10
Precision, Quantization & the Compute-Memory Tradeoff
7.11
Accelerator Selection, TCO & Procurement Strategy
7.12
On-Package Power Delivery & Power Integrity
7.13
The Rack as Integration Unit
7.14
Server & System Integration
7.15
Deployment Velocity & Cabling at Scale
8
Networking, Fabrics & Optics
8.1
Network Fundamentals & AI Traffic Characterization
8.2
Scale-Up Fabric (Intra-Node / Intra-Rack)
8.3
Network Silicon: Switch ASICs, NICs & DPUs
8.4
Scale-Out Fabric: Protocols, Standards & Transport
8.5
Scale-Out Topology, Sizing & Oversubscription
8.6
Congestion Control, Load Balancing & In-Network Compute
8.7
Management, Out-of-Band Fabric & PTP/IEEE-1588 Timing
8.8
Scale-Across: Multi-Campus & Cross-Region Fabric (DCI for Distributed Training)
8.9
Physical-Layer & Interconnect Taxonomy
8.10
CPO, Fiber Plant & Structured Cabling
9
Storage & Data
9.1
Storage in the AI Lifecycle: Why It Determines GPU Efficiency
9.2
Parallel & Distributed File Systems
9.3
NVMe Tiers, GPUDirect Storage & the CPU-Bypass Data Path
9.4
Checkpointing for Large-Scale Training
9.5
Data Ingestion, Preprocessing & the Data-Loader Path
9.6
Object Storage, Data Lakes & the Capacity Tier
9.7
Inference & KV-Cache Storage: The New Memory Hierarchy
9.8
Sizing, Data Gravity & Resilience
9.9
The Data-Prep Supercomputer: Offline Data Processing
10
Software, Orchestration & Service Delivery
10.1
Orchestration Architecture & the Scheduling Plane
10.2
Topology-Aware & Rack-Scale Scheduling
10.3
Multi-Tenancy, Isolation & Resource Sharing
10.4
Node Software Stack: Drivers, CUDA/ROCm, NCCL & Firmware
10.5
Provisioning, Bring-Up & Infrastructure as Code
10.6
Observability, Telemetry & GPU Health
10.7
Fleet Reliability, Fault Tolerance & Autonomous Recovery
10.8
MLOps & Training Frameworks
10.9
Customer Onboarding, Delivery & Productization
10.10
Data Governance, Privacy & the Training-Data Legal Regime
10.11
Inference Serving Engineering: SLOs, Batching, Disaggregation & Goodput-Optimal Scheduling
11
Security
11.1
Threat Model, Assets & Security Levels for AI Infrastructure
11.2
Physical Security: Siting, Zones & Kinetic/Drone Threats
11.3
Supply-Chain Security & Hardware Provenance
11.4
Hardware Root of Trust, Firmware & BMC Security
11.5
GPU Confidential Computing & Trusted Execution
11.6
Multi-Tenant & Workload Isolation Security
11.7
Network Segmentation, Microsegmentation & Zero Trust
11.8
Model & Weight Protection (At-Rest, In-Transit, In-Use)
11.9
Insider Threat & Human-Layer Security
11.10
Cyber-Physical & Destructive Attacks on OT/Facility Systems
11.11
Compliance, Certification & Governance
11.12
Security Operations, Detection & Incident Response
12
Reliability, Resilience & Standards
12.1
Resilience Standards, Redundancy Topologies & Fault-Domain Engineering
12.2
The AI-Cluster Reliability Rethink: Goodput vs Facility Availability
12.3
Disaster Recovery, Business Continuity & Geographic Failover
12.4
SLAs, Goodput Contracts & Availability Commitments
12.5
Quantitative Reliability & Availability Modeling (RBD / FTA / Monte-Carlo)
13
Commissioning & Go-Live
13.1
Commissioning Fundamentals, Levels & Program Governance
13.2
Documentation, Scripts & Acceptance Test Plans
13.3
Electrical Power Acceptance (L3/L4)
13.4
Commissioning On-Site Generation & Microgrid Controls
13.5
Cooling Acceptance: Air, Liquid-to-Chip & CDU Commissioning
13.6
Level 5 Integrated Systems Testing (IST) & Failure-Mode Demonstration
13.7
Network Fabric Commissioning & Validation
13.8
GPU Node Burn-In, Diagnostics & Stress Validation
13.9
Cluster-Scale Benchmarking, Reference Training & Storage/Scheduler Validation
13.10
Staged Power/Load Ramp, Go-Live & Handover to Operations
14
Day-2 Operations, Upgrades & Lifecycle
14.1
Operational KPIs, Goodput & the Reliability Economics of AI Factories
14.2
DCIM, Telemetry & Observability for GPU-Dense, Liquid-Cooled Facilities
14.3
Component Failure Modes, Failure Rates & Fleet Reliability Data
14.4
Reliability Engineering for Training (Operational)
14.5
Predictive & Preventive Maintenance of Power and Cooling Plant
14.6
Spares Strategy, RMA Logistics & Repair Operations
14.7
Capacity, Power & Thermal Management in Operation
14.8
Firmware & Software Lifecycle Management at Fleet Scale
14.9
Hardware Refresh, Depreciation Strategy, Decommissioning & ITAD
14.10
Facility Decommissioning, Repowering & Site Remediation
14.11
Operations Organization, Workforce, Talent & Incident Command
14.12
Operational Procedures, Change Management & Human-Error Control
14.13
Agentic Ops, RL Control & the Autonomy Ladder
14.14
Continuous & Re-Commissioning on a Live Campus
15
Sustainability & Efficiency
15.1
Efficiency Metrics: PUE, WUE, ERF, REF & the Post-PUE Metric Stack
15.2
Energy Efficiency: Cooling, Free Cooling, Setpoints & Power-Chain Losses
15.3
Carbon, Clean Power Procurement & 24/7 Carbon-Free Energy
15.4
Water Stewardship
15.5
Heat Reuse & District Heating (Sustainability & Economics)
15.6
Embodied Carbon & Circularity Across the Lifecycle
15.7
Regulation, Reporting & Disclosure Frameworks
15.8
Grid Impact, Energy-Systems Integration & Grid Services
16
Trends, Roadmaps & the Future
16.1
The Power-Bound Era: Why the Bottleneck Moved to the Substation
16.2
Subsystem Roadmaps 2026 → 2030 (Consolidated)
16.3
Software, Orchestration & Efficiency at the Frontier
16.4
The Economics of the Build-Out
16.5
Scenarios for 2030
§
Appendices & Reference Data
A
Standards & Specifications Cross-Reference Matrix
B
Reference Designs & Worked Examples
C
Decision Tables & Calculators
D
Numbers Provenance & Forecast Register
E
Glossary, Phase-Gate Timeline & Learning/Community Map
F
Failure-Mode / FMEA Catalog
G
Regional & International Design Deltas: Consolidated Quick-Reference Crosswalk
Guide
›
Glossary
›
DCGM
DCGM
· Data Center GPU Manager
NVIDIA's tool for monitoring, diagnosing and health-checking GPUs across a fleet.
← All terms