H100
NVIDIA's Hopper-generation data-center GPU; the chip behind most of the 2023–2024 buildout wave and the baseline many density and power comparisons start from.
Current numbers
$1.90/M output tokens (illustrative node rental)illustrative H100 node-rental cost for self-hosted 70B inference; calculation in Chapter 1.8
~40 → 600 kWper-rack power roadmap: H100 ~40 kW (2023), GB200 NVL72 ~132 kW nominal (2024), Kyber ~600 kW (NVIDIA targets H2 2027; SemiAnalysis reported 2028 in July 2026)
80 → 288 GB → ~1 TBHBM capacity per accelerator package: H100 80 GB; HGX B200 180 GB; B300 288 GB (HBM3E); Rubin Ultra ~1 TB HBM4E on a four-chiplet package (roadmap)