NVIDIA Accelerators: Hopper → Blackwell → Vera Rubin → Rubin Ultra → Feynman

NVIDIA sells a cadence: an annual rhythm of accelerator generations, each one re-drawing the rack, the fabric, and the power chain underneath it. The question it forces is not which GPU is fastest but which rung of the ramp you are committing your building to — and what the next rung costs if you guessed wrong. Since 2022 the unit of purchase has migrated upward: from the H100 as a board, to the HPE GB200 NVL72 profile as a 132 kW rack you buy whole, to the Vera Rubin NVL72 and the Rubin Ultra NVL576 / Kyber NVL144 generation as multi-rack pods plumbed for 800 VDC. Each migration can commit the facility to a new interface; a roadmap announcement is neither a delivery promise nor an accepted rack. You can defer the silicon; you cannot defer the floor loading, the water, and the interconnection slot the silicon implies.

The roadmap here reads as a sequence of decisions and their downstream costs. The per-GPU specs run across Hopper → Blackwell → Blackwell Ultra → Vera Rubin → Rubin Ultra → Feynman; the NVL system — not the GPU — became the unit of procurement once the size of the scale-up domain you buy (8 → 72 → 144 → 576 GPUs) turned into a strategic commitment rather than a datasheet line; the disaggregated-inference fork that Rubin CPX opened is now answered by Groq's LPU/LPX decode racks; and the annual roadmap cadence compresses competitors' design windows and can pressure your economic refresh schedule without changing book life. Per-GPU NVLink bandwidth appears here as a datasheet attribute; the NVLink/NVSwitch fabric that aggregates it has its canonical home in Chapter 8.2.

The instinct is to compare generations on peak FLOPS. That is the marketing-number trap (Chapter 7.1): the headline figures are sparse FP4 with all the asterisks stripped, and they tell you almost nothing about what you must build. The number that actually cascades through your facility is rack power. An H100 air-cooled rack lands near 40 kW; the HPE GB200 NVL72 profile is rated 132 kW and mandates direct-to-chip liquid; a GB300 NVL72 carries an up-to-142 kW facility design basis; the Vera Rubin VR200 NVL72 carries a 330 kW cabinet facility design basis; and Rubin Ultra's Kyber rack (NVL144) targets ~600 kW on an 800 VDC bus. Those are different OEM operating profiles, facility allowances and roadmaps, not a 15× measured power trajectory.

A hall scoped for the previous generation needs an interface-by-interface retrofit check. You do not get to "upgrade" from a 40 kW air hall to a 132 kW liquid hall by swapping racks — the floor loading must be checked against the selected OEM’s populated operating mass and point loads, the airflow, electrical headroom and facility water must each satisfy the selected profile. The density wall (Chapter 5.1) and the DLC default (Chapter 5.4) are downstream of which rung of this ramp you bought into. The power curve governs; the compute curve follows.

Hopper (H100, 2022 / H200, 2024) is the generation most of the installed base still runs. H100 ships 80 GB HBM3 at ~3.35 TB/s, ~700 W TDP, FP8 Transformer Engine, NVLink 4 at 900 GB/s per GPU. H200 is the same compute die with 141 GB HBM3E at ~4.8 TB/s — a memory-bandwidth refresh that disproportionately helps inference decode. Hopper is air-coolable, which is exactly why it became the default and why the jump to Blackwell broke so many facility assumptions.

Blackwell (B200 / GB200, 2024–2025) is a dual-die GPU — two reticle-limited compute dies on one package behaving as a single CUDA device over a 10 TB/s die-to-die link — with 180 GB of usable HBM3E (HGX B200; NVIDIA's current GB200 specification gives 372 GB across the superchip's two GPUs — 186 GB usable per Blackwell GPU and ~13.4 TB across an NVL72, against the 192 GB raw figure earlier material quoted), a second-generation Transformer Engine adding native FP4, and NVLink 5 at 1.8 TB/s per GPU. The GB200 superchip pairs two Blackwell GPUs with one Grace CPU over NVLink-C2C. Blackwell Ultra (B300 / GB300, 2025) lifts HBM to 288 GB and adds steady-power and transient-mitigation features (capacitor energy storage, ramp smoothing) that exist because a ~135 kW-TDP / 155 kW-peak rack toggling between idle and full all-reduce is a grid problem (Chapter 7.12).

Vera Rubin (H2 2026) is the next platform, not just a chip — "VR200" is supply-chain/roadmap shorthand, not NVIDIA's spelled-out name. The Rubin GPU is again dual-die on a 4-reticle CoWoS-L interposer — ~336 billion transistors, 1.6x Blackwell — with 288 GB HBM4 across 8 stacks at up to ~22 TB/s, sixth-generation Tensor Cores, and NVLink 6 at 3.6 TB/s per GPU. The Vera CPU is NVIDIA's custom Arm successor to Grace. The rack-scale unit is the NVL72 — 72 Rubin GPUs and 36 Vera CPUs (NVIDIA briefly marketed it as "NVL144" by counting the 144 compute dies inside the 72 dual-die packages, then reverted to the package-based NVL72 name at CES/GTC 2026), delivering ~3.3x the GB300 NVL72 on inference, ~3.6 EF FP4 inference / ~1.2 EF FP8 training per rack, with ~260 TB/s of scale-up NVLink bandwidth. NVIDIA says the Vera Rubin platform is in full production and partner availability is planned for the second half of 2026.

Rubin Ultra (H2 2027) is where the unit of purchase jumps again. As shown at GTC 2026 it packs four compute dies per package (~100 PFLOPS FP4, 1 TB HBM4e per package) and deploys two ways: the ~600 kW Kyber rack — NVL144: 144 quad-die packages = 576 GPU compute dies on 800 VDC, ~15 EF FP4 inference / ~5 EF FP8 training, with eight Kyber racks forming NVL1152 — and the NVL576 system, eight MGX NVL72-class racks stitched into one 576-GPU optical NVLink domain. (Supply-chain reports from Apr–Jun 2026 say Rubin Ultra was scaled back to a dual-die "2+2" Kyber board layout over CoWoS-L warpage and yield — unconfirmed by NVIDIA, so the per-package quad-die specs are in flux; the ~600 kW rack-level power and H2 2027 timeline are unaffected.) Feynman (2028) is the next architecture on the roadmap — paired with the new Rosa CPU, advanced 3D die stacking, and NVLink 8, with the ConnectX/Spectrum generations advancing in lockstep; it is widely expected (though not confirmed by NVIDIA) to move to a sub-2 nm TSMC node with backside power delivery. The public roadmap targets an approximately annual platform cadence; dates, product configuration and volume availability remain roadmap commitments until shipped and independently verified.

The rack-power column governs the table, not the FLOPS column. The compute numbers grow impressively, but they are the easy part — their delivery and packaging schedule is itself a material risk. What strands capital is the rightmost columns: the rack unit changes shape (board → 72-GPU rack → 144 → 576), the power per rack escalates an order of magnitude, and the cooling and voltage architecture flip underneath. Moving between generations requires a gap analysis across rack geometry, mass/reactions, power topology and transients, coolant/TCS/FWS conditions, residual air, fabric, controls, safety and service routes. Reserved capacity can reduce work, but neither a one-decision upgrade nor demolition is universal. The generation you choose is the substrate you commit to.

The Rubin line crossed from roadmap to silicon in mid-2026: CoreWeave brought up and validated the industry-first Vera Rubin NVL72 rack on June 1, 2026 (Livingston, NJ; Dell PowerEdge XE9812-integrated) — 72 Rubin GPUs plus 36 Vera CPUs, ~3.6 EFLOPS FP4, 260 TB/s NVLink 6 all-to-all — and its power basis is now published: Pegatron's RA4803-72N3 specifies 188 kW (Max Q) / 228 kW (Max P), while NVIDIA DSX sets a 330 kW cabinet TDP facility design basis. Size irreversible infrastructure to the facility design basis; run energy and TCO models on the chosen operating profile.

Through Hopper, the unit was the 8-GPU HGX board and the scale-up domain was 8 GPUs wide. Blackwell broke that model: the GB200 NVL72 fuses 72 Blackwell GPUs and 36 Grace CPUs into a single NVLink domain — 18 compute trays and 9 NVSwitch trays connected by a copper NVLink spine carrying ~130 TB/s of aggregate rack bandwidth across more than 5,000 in-rack copper cables — so that every GPU has load/store addressability over NVLink to ~13.4 TB of distributed HBM across the domain; remote access does not make every GPU's memory local, uniform-latency or universally cache-coherent (~1.44 EF FP4 sparse). The integrated rack is the procurement and acceptance boundary, including when an OEM assembles it from qualified parts. The reason this matters strategically is that the scale-up domain size is now a purchasing decision that sets your parallelism ceilings until your next refresh.

The consequence runs in both directions. A wide domain (72 → 144 → 576 GPUs) lets you fit tensor-parallel and pipeline-parallel groups, and especially wide expert-parallel MoE inference, entirely inside the NVLink fabric — where per-direction bandwidth is roughly 9x the scale-out NIC on a current 800G port and 18x on a 400G one — instead of spilling collectives onto the slower back-end network. Wide-EP MoE serving (e.g., EP32 vs EP8) is the canonical workload that the big domain unlocks (Chapter 8.2). But a wide domain you do not use is stranded capital: a latency-bound 8B-parameter inference service pinned to a 72-GPU NVLink rack is paying for a fabric it never lights up. Buy the smallest supported domain that the working set and measured collectives require — the NVL system is a commitment, not a default.

Rubin introduces a second, quieter fork that reshapes the inference BOM. Long-context inference has two phases with opposite hardware profiles: the context (prefill) phase reads and encodes the entire input — compute-bound, hungry for FLOPS, light on memory bandwidth — while the generation (decode) phase emits tokens one at a time, memory-bandwidth-bound and latency-sensitive, leaning on HBM and the KV cache. A monolithic GPU sized for decode (expensive HBM) is overpaying to do prefill; a GPU sized for prefill is starved on memory for decode. NVIDIA's original answer was Rubin CPX, announced 2025-09-09 — a context-phase accelerator with ~30 PFLOPS NVFP4, 3x attention acceleration over GB300, and 128 GB of GDDR7 rather than HBM, roughly 5x more cost-effective per byte than HBM for this compute-bound role.

CPX's roadmap status is now uncertain. Credible secondary reporting — The Next Platform quoting NVIDIA's Ian Buck, corroborated by Tom's Hardware and The Elec — says CPX was reportedly removed from the roadmap at GTC 2026 (March), before production silicon and about six months after its September 2025 announcement. NVIDIA issued no official cancellation and has not reaffirmed CPX on a current roadmap page, so treat it as announced, roadmap status uncertain / reportedly pulled — not as shipping, and not as formally cancelled. In the reported reshuffle the context/prefill role does not move to a dedicated chip: Rubin GPUs handle prefill and attention themselves, and the latency-critical decode phase is reassigned to Groq silicon under NVIDIA's ~$20B Groq deal.

The two Groq products are easy to conflate, so name them precisely. The chip is the Groq 3 LPU — an SRAM-based inference processor with ~500 MB of on-chip SRAM and ~150 TB/s of SRAM bandwidth, no HBM, that cannot compute prefill attention. The rack is the LPX: 256 LPUs with ~128 GB of aggregate on-chip SRAM, the deployable unit that runs the token-by-token decode loop (the FFN/MoE layers) while the KV cache and attention stay on the Rubin HBM GPUs. (One secondary outlet speculates a CPX-style context accelerator could resurface with Feynman around 2028 — speculation, not an NVIDIA statement.)

A commercial-structure footnote, not a silicon change (Aug 2026): the post-acquisition Groq entity announced it is an NVIDIA Cloud Partner (Groq, 2026-08-12) — buying B300/GB300/Rubin GPUs alongside LPUs and renting the capacity out, with SemiAnalysis noting many of the clusters Groq rents are straight Blackwell with no LPUs at all. Groq-as-neocloud now coexists with the LPU decode path described above; nothing in NVIDIA's CPX/LPU integration story has officially changed.

The decision: for million-token-context workloads, do you adopt disaggregated serving — a pool of Rubin (HBM) GPUs doing prefill and attention feeding a pool of SRAM-based Groq LPX decode racks doing low-latency generation (FFN/MoE), coupled over the fabric by streaming intermediate activations between the attention and FFN stages — or stay monolithic? The KV cache stays on the Rubin GPUs, so this is not the KV-handoff transport of a classic prefill–decode split. Disaggregation wins on cost-per-token and interactive latency at long context because you run the token-by-token decode on Groq's cheap, deterministic SRAM silicon instead of monopolizing HBM GPUs for generation; it costs you a more complex serving stack (separate pools, KV-cache transport, careful ratio tuning) and a fabric that must move KV cache between phases efficiently (Chapter 10.11). For short-context, latency-flat workloads the disaggregation overhead does not pay. But the fork is wider than long context: NVIDIA describes prefill–decode, attention–FFN and external-drafter arrangements, and each imposes a different traffic volume and latency requirement — name the arrangement before you derive a fabric requirement from it.

The cadence is a competitive instrument as much as a delivery schedule, and it cuts two ways. Against competitors, a yearly architecture compresses the window any challenger has to close a gap: if AMD or a custom-ASIC roadmap matches Blackwell as a qualified Rubin system arrives, the comparison resets (Chapter 7.3, Chapter 7.5). The software moat (CUDA, the NCCL/Dynamo/TensorRT stack) compounds this — a one-year hardware tick gives the ecosystem a fresh target every twelve months. Against the buyer, the same cadence is a depreciation accelerant: a frontier accelerator's economic life is tested at 2–3 years in the guide's contested bear case, against published 4–6-year useful-life estimates and 5–6-year book policies, because next year's part does the same work at materially lower cost-per-token. The cadence that protects NVIDIA's lead also shortens your amortization runway.

That sets up the buyer's real decision — ride every generation, or skip? Riding each step can improve qualified output-per-dollar but means continuous capital outlay and the operational churn of new power, cooling, and fabric envelopes every year. Skipping a generation (e.g., Hopper → Rubin, bypassing Blackwell) reduces churn and lets one substrate investment serve longer, at the cost of running a generation behind on token economics during the gap. The deciding variables are your residual-value assumption (do used GPUs hold enough value to backstop the refresh — see Chapter 1.8) and whether your facility substrate can even accept the generation you would skip to. A 40 kW air-cooled position cannot accept a roadmap 600 kW Kyber profile without facility work. An announcement offers a refresh option: exercise it only when Chapter 1.8 covers hardware, migration, facility work, downtime and the value of keeping the fleet.

A defensible accelerator strategy uses the same discipline that governs facility scoping (Chapter 1.1): sort the decisions by the cost of changing your mind. The roadmap makes some things reversible and some irreversible, and they are not the ones people assume.

• Reversible (defer, re-decide at refresh): the specific accelerator generation within a power/cooling envelope — a hall plumbed for ~140 kW liquid still needs the exact GB200 or GB300 electrical, liquid, room-air and service profile checked; the HGX-vs-NVL choice for an inference fleet; the disaggregation decision for inference serving; the ride-vs-skip refresh cadence.
• Irreversible (commit at scoping): whether the hall is plumbed for liquid at all; the floor-loading basis for 3,000-lb-plus wet racks; the electrical capacity and voltage path (415/480 VAC vs an 800 VDC future); reserved physical headroom — busbar runs, pipe racks, switchgear room — for the next density step; and Rubin readiness, because VR200 NVL72 at a 330 kW cabinet facility design basis is a substrate step rather than a rack swap. Price the option premium now against later retrofit cost; a reservation is not automatically cheap.

The strategic move is the same as everywhere in this guide: convert irreversible decisions into reversible ones while the option is cheap. Reserve the busbar capacity and water for a density step-up you have not committed to; buy into the NVL domain your workload uses today but provision the substrate for the domain you might buy in two years. Design the rungs you can reach into the building before you need them; Chapter 7.13 records manufacturer, revision, mode, interfaces and enabling works, and Chapter 1.8 prices the refresh.