NVIDIA Accelerators: Hopper → Blackwell → Vera Rubin → Rubin Ultra → Feynman

NVIDIA sells a cadence: an annual rhythm of accelerator generations, each one re-drawing the rack, the fabric, and the power chain underneath it. The question it forces is not which GPU is fastest but which rung of the ramp you are committing your building to — and what the next rung costs if you guessed wrong. Since 2022 the unit of purchase has migrated upward: from the H100 as a board, to the GB200 NVL72 as a 132 kW rack you buy whole, to the Vera Rubin NVL72 and the Rubin Ultra NVL576 / Kyber NVL144 generation as multi-rack pods plumbed for 800 VDC. Each migration is a one-way door for the facility that hosts it. You can defer the silicon; you cannot defer the floor loading, the water, and the interconnection slot the silicon implies.

The roadmap here reads as a sequence of decisions and their downstream costs. The per-GPU specs run across Hopper → Blackwell → Blackwell Ultra → Vera Rubin → Rubin Ultra → Feynman; the NVL system — not the GPU — became the unit of procurement once the size of the scale-up domain you buy (8 → 72 → 144 → 576 GPUs) turned into a strategic commitment rather than a datasheet line; the disaggregated-inference fork that Rubin CPX opened is now answered by Groq's LPU/LPX decode racks; and the annual cadence compresses competitors' design windows and your own depreciation schedule at once. Per-GPU NVLink bandwidth appears here as a datasheet attribute; the NVLink/NVSwitch fabric that aggregates it has its canonical home in Chapter 8.2.

The instinct is to compare generations on peak FLOPS. That is the marketing-number trap (Chapter 7.1): the headline figures are sparse FP4 with all the asterisks stripped, and they tell you almost nothing about what you must build. The number that actually cascades through your facility is rack power. An H100 air-cooled rack lands near 40 kW; a GB200 NVL72 draws ~132 kW and mandates direct-to-chip liquid; a GB300 NVL72 pushes ~140 kW; the Vera Rubin VR200 NVL72 lands in the ~190–230 kW band; and Rubin Ultra's Kyber rack (NVL144) targets ~600 kW on an 800 VDC bus. That is a 15x escalation in rack power across roughly four years.

A hall scoped for the previous generation's density cannot absorb the next one without a substrate it does not have. You do not get to "upgrade" from a 40 kW air hall to a 132 kW liquid hall by swapping racks — the floor loading is wrong (a wet NVL72 is ~1.36 t / 3,000 lb), the plenum is wrong, the electrical headroom is wrong, and there is no facility water. The density wall (Chapter 5.1) and the DLC default (Chapter 5.4) are downstream of which rung of this ramp you bought into. The power curve governs; the compute curve follows.

Hopper (H100, 2022 / H200, 2024) is the generation most of the installed base still runs. H100 ships 80 GB HBM3 at ~3.35 TB/s, ~700 W TDP, FP8 Transformer Engine, NVLink 4 at 900 GB/s per GPU. H200 is the same compute die with 141 GB HBM3E at ~4.8 TB/s — a memory-bandwidth refresh that disproportionately helps inference decode. Hopper is air-coolable, which is exactly why it became the default and why the jump to Blackwell broke so many facility assumptions.

Blackwell (B200 / GB200, 2024–2025) is a dual-die GPU — two reticle-limited compute dies on one package behaving as a single CUDA device over a 10 TB/s die-to-die link — with 180 GB of usable HBM3E (HGX B200; the GB200 superchip is specced at 192 GB per Blackwell GPU), a second-generation Transformer Engine adding native FP4, and NVLink 5 at 1.8 TB/s per GPU. The GB200 superchip pairs two Blackwell GPUs with one Grace CPU over NVLink-C2C. Blackwell Ultra (B300 / GB300, 2025) lifts HBM to 288 GB and adds steady-power and transient-mitigation features (capacitor energy storage, ramp smoothing) that exist because a 140 kW rack toggling between idle and full all-reduce is a grid problem (Chapter 7.12).

Vera Rubin (H2 2026) is the next platform, not just a chip — "VR200" is supply-chain/roadmap shorthand, not NVIDIA's spelled-out name. The Rubin GPU is again dual-die on a 4-reticle CoWoS-L interposer — ~336 billion transistors, 1.6x Blackwell — with 288 GB HBM4 across 8 stacks at up to ~22 TB/s, sixth-generation Tensor Cores, and NVLink 6 at 3.6 TB/s per GPU. The Vera CPU is NVIDIA's custom Arm successor to Grace. The rack-scale unit is the NVL72 — 72 Rubin GPUs and 36 Vera CPUs (NVIDIA briefly marketed it as "NVL144" by counting the 144 compute dies inside the 72 dual-die packages, then reverted to the package-based NVL72 name at CES/GTC 2026), delivering ~3.3x the GB300 NVL72 on inference, ~3.6 EF FP4 inference / ~1.2 EF FP8 training per rack, with ~260 TB/s of scale-up NVLink bandwidth. NVIDIA declared the Vera Rubin platform in full production at GTC 2026 (March 16, 2026), with rack production shipments beginning fall 2026.

Rubin Ultra (H2 2027) is where the unit of purchase jumps again. As shown at GTC 2026 it packs four compute dies per package (~100 PFLOPS FP4, 1 TB HBM4e per package) and deploys two ways: the ~600 kW Kyber rack — NVL144: 144 quad-die packages = 576 GPU compute dies on 800 VDC, ~15 EF FP4 inference / ~5 EF FP8 training, ~365 TB total memory, with eight Kyber racks forming NVL1152 — and the NVL576 system, eight MGX NVL72-class racks stitched into one 576-GPU optical NVLink domain. (Supply-chain reports from Apr–Jun 2026 say Rubin Ultra was scaled back to a dual-die "2+2" Kyber board layout over CoWoS-L warpage and yield — unconfirmed by NVIDIA, so the per-package quad-die specs are in flux; the ~600 kW rack-level power and H2 2027 timeline are unaffected.) Feynman (2028) is the next architecture on the roadmap — paired with the new Rosa CPU, advanced 3D die stacking, and NVLink 8, with the ConnectX/Spectrum generations advancing in lockstep; it is widely expected (though not confirmed by NVIDIA) to move to a sub-2 nm TSMC node with backside power delivery. The cadence is explicit and locked: a new architecture every year.

The rack-power column governs the table, not the FLOPS column. The compute numbers grow impressively, but they are the easy part — TSMC and HBM deliver them on schedule. What strands capital is the rightmost columns: the rack unit changes shape (board → 72-GPU rack → 144 → 576), the power per rack escalates an order of magnitude, and the cooling and voltage architecture flip underneath. A facility that bought into Blackwell at 132 kW and liquid cooling is one substrate decision (reserved busbar capacity, water headroom, floor loading) away from Rubin; a facility that bought into Hopper at 40 kW and air is a demolition-and-rebuild away. The generation you choose is the substrate you commit to.

The Rubin line crossed from roadmap to silicon in mid-2026: CoreWeave brought up and validated the industry-first Vera Rubin NVL72 rack on June 1, 2026 (Livingston, NJ; Dell PowerEdge XE9812-integrated) — 72 Rubin GPUs plus 36 Vera CPUs, ~3.6 EFLOPS FP4, 260 TB/s NVLink 6 all-to-all — but that is a single validated rack, not fleet-scale general availability, and the ~190 kW (Max Q) / ~230 kW (Max P) rack-power figures the table quotes are analyst-sourced (Kuo supply-chain checks), not an NVIDIA datasheet. The substrate is now provable in silicon; the power budget you design to is still an estimate.

Through Hopper, the unit was the 8-GPU HGX board and the scale-up domain was 8 GPUs wide. Blackwell broke that model: the GB200 NVL72 fuses 72 Blackwell GPUs and 36 Grace CPUs into a single NVLink domain — 18 compute trays and 9 NVSwitch trays connected by a copper NVLink spine carrying ~130 TB/s of aggregate rack bandwidth across more than 5,000 in-rack copper cables — so that all 72 GPUs address each other at full NVLink speed as one coherent memory fabric (~13.4 TB of unified memory, ~1.44 EF FP4 sparse). You do not assemble this from parts; you buy the rack as a SKU. The reason this matters strategically is that the scale-up domain size is now a purchasing decision that sets your parallelism ceilings until your next refresh.

The consequence runs in both directions. A wide domain (72 → 144 → 576 GPUs) lets you fit tensor-parallel and pipeline-parallel groups, and especially wide expert-parallel MoE inference, entirely inside the NVLink fabric — where bandwidth is ~5–10x the scale-out NIC — instead of spilling collectives onto the slower back-end network. Wide-EP MoE serving (e.g., EP32 vs EP8) is the canonical workload that the big domain unlocks (Chapter 8.2). But a wide domain you do not use is stranded capital: a latency-bound 8B-parameter inference service pinned to a 72-GPU NVLink rack is paying for a fabric it never lights up. Buy the domain the workload consumes — the NVL system is a commitment, not a default.

Rubin introduces a second, quieter fork that reshapes the inference BOM. Long-context inference has two phases with opposite hardware profiles: the context (prefill) phase reads and encodes the entire input — compute-bound, hungry for FLOPS, light on memory bandwidth — while the generation (decode) phase emits tokens one at a time, memory-bandwidth-bound and latency-sensitive, leaning on HBM and the KV cache. A monolithic GPU sized for decode (expensive HBM) is overpaying to do prefill; a GPU sized for prefill is starved on memory for decode. NVIDIA's original answer was Rubin CPX, announced 2025-09-09 — a context-phase accelerator with ~30 PFLOPS NVFP4, 3x attention acceleration over GB300, and crucially 128 GB of GDDR7 rather than HBM, roughly 5x more cost-effective per byte than HBM for this compute-bound role.

CPX's roadmap status is now uncertain. Credible secondary reporting — The Next Platform quoting NVIDIA's Ian Buck, corroborated by Tom's Hardware and The Elec — says CPX was reportedly removed from the roadmap at GTC 2026 (March), before production silicon and about six months after its September 2025 announcement. NVIDIA issued no official cancellation and has not reaffirmed CPX on a current roadmap page, so treat it as announced, roadmap status uncertain / reportedly pulled — not as shipping, and not as formally cancelled. In the reported reshuffle the context/prefill role does not move to a dedicated chip: Rubin GPUs handle prefill and attention themselves, and the latency-critical decode phase is reassigned to Groq silicon under NVIDIA's ~$20B Groq deal.

The two Groq products are easy to conflate, so name them precisely. The chip is the Groq 3 LPU — an SRAM-based inference processor with ~500 MB of on-chip SRAM and ~150 TB/s of SRAM bandwidth, no HBM, that cannot compute prefill attention. The rack is the LPX: 256 LPUs with ~128 GB of aggregate on-chip SRAM, the deployable unit that runs the token-by-token decode loop (the FFN/MoE layers) while the KV cache and attention stay on the Rubin HBM GPUs. (One secondary outlet speculates a CPX-style context accelerator could resurface with Feynman around 2028 — speculation, not an NVIDIA statement.)

The decision: for million-token-context workloads, do you adopt disaggregated serving — a pool of Rubin (HBM) GPUs doing prefill and attention feeding a pool of SRAM-based Groq LPX decode racks doing low-latency generation (FFN/MoE), coupled over the fabric via KV-cache transfer — or stay monolithic? Disaggregation wins on cost-per-token and interactive latency at long context because you run the token-by-token decode on Groq's cheap, deterministic SRAM silicon instead of monopolizing HBM GPUs for generation; it costs you a more complex serving stack (separate pools, KV-cache transport, careful ratio tuning) and a fabric that must move KV cache between phases efficiently (Chapter 10.11). For short-context, latency-flat workloads the disaggregation overhead does not pay — this is a long-context-specific fork.

The cadence is a competitive instrument as much as a delivery schedule, and it cuts two ways. Against competitors, a yearly architecture compresses the window any challenger has to close a gap: by the time an AMD or a custom-ASIC roadmap matches Blackwell, Rubin is shipping, and the comparison resets (Chapter 7.3, Chapter 7.5). The software moat (CUDA, the NCCL/Dynamo/TensorRT stack) compounds this — a one-year hardware tick gives the ecosystem a fresh target every twelve months. Against the buyer, the same cadence is a depreciation accelerant: a frontier accelerator's economic life is now 2–3 years against a 5–6 year book life, because next year's part does the same work at materially lower cost-per-token. The cadence that protects NVIDIA's lead also shortens your amortization runway.

That sets up the buyer's real decision — ride every generation, or skip? Riding each step maximizes performance-per-dollar-per-token but means continuous capital outlay and the operational churn of new power, cooling, and fabric envelopes every year. Skipping a generation (e.g., Hopper → Rubin, bypassing Blackwell) reduces churn and lets one substrate investment serve longer, at the cost of running a generation behind on token economics during the gap. The deciding variables are your residual-value assumption (do used GPUs hold enough value to backstop the refresh — see Chapter 7.11) and whether your facility substrate can even accept the generation you would skip to. You cannot skip from a 40 kW air hall to a 600 kW Kyber pod; the skip is only available if you provisioned the substrate for it.

A defensible accelerator strategy uses the same discipline that governs facility scoping (Chapter 1.1): sort the decisions by the cost of changing your mind. The roadmap makes some things reversible and some irreversible, and they are not the ones people assume.

• Reversible (defer, re-decide at refresh): the specific accelerator generation within a power/cooling envelope — a hall plumbed for 132 kW liquid can take GB200, GB300, and likely an early Rubin SKU without re-architecting; the HGX-vs-NVL choice for an inference fleet; the disaggregation decision for inference serving; the ride-vs-skip refresh cadence.
• Irreversible (commit at scoping): whether the hall is plumbed for liquid at all; the floor-loading basis for 3,000-lb-plus wet racks; the electrical capacity and voltage path (415/480 VAC vs an 800 VDC future); and reserved physical headroom — busbar runs, pipe racks, switchgear room — for the next density step. These are the decisions where the option premium is cheap to pay now and very expensive to retrofit later.

The strategic move is the same as everywhere in this guide: convert irreversible decisions into reversible ones while the option is cheap. Reserve the busbar capacity and water for a density step-up you have not committed to; buy into the NVL domain your workload uses today but provision the substrate for the domain you might buy in two years. Design the rungs you can reach into the building before you need them.