Calculators › Cluster sizing — model → MW

Cluster sizing — model → MW

Size an inference fleet end to end: model parameters, quantization, KV cache and context length → VRAM per replica → GPUs, replicas for your token demand, racks, facility megawatts, and monthly cost.

Cluster sizing — model to megawatts

Model sizeB paramsBytes / param (FP16 2 · FP8 1 · INT4 0.5)KV cache / token (Llama 70B ≈ 330 · 405B ≈ 516)KBContext lengthtokConcurrent requests / replicaActivation + framework overhead%HBM per GPU (H100 80 · B200 192 · GB300 288)GBPeak demandtok/sThroughput / replicatok/sTarget utilization (headroom for bursts)%GPUs / rack (NVL72 = 72 · HGX = 32)GPU TDPWPUERental price$/GPU-hr

130 GPUs · 0.2 MW

VRAM per replica (weights 70 + KV 676 GB)858 GB

GPUs per replica → replicas for demand5 × 26

Racks (72/rack)2

Facility draw (incl. host + PUE)0.2 MW

Fleet rental cost$217,321/mo

Weights are the floor — at long context the KV cache dominates VRAM (70–90% at 1M tokens → Ch 9.7). Tensor-parallel degree prefers powers of two; serving engineering (batching, disaggregation) moves throughput ~10× → Ch 10.11.

All models are open — change any default to your case, then save, share a permalink, or export to CSV. See the full calculator suite.