Prefill
The compute-heavy phase that processes an inference prompt and builds its KV cache before generation begins.
Current numbers
GTC 2026 (reported)Groq 3 LPU chip (~500 MB SRAM, ~150 TB/s) → LPX rack (256 LPUs, ~128 GB aggregate SRAM) fills the disaggregated decode/FFN slot (no HBM); Rubin GPUs retain prefill + attention. Rubin CPX reportedly pulled at GTC 2026 (no official NVIDIA cancellation)
xPyDruntime-reconfigurable disaggregation: x prefill workers feeding y decode workers, re-balanced live