Decode
In the data pipeline: turning stored, compressed samples (images, video, packed text shards) back into training-ready tensors — CPU-bound work that gates GPU utilization.
Context: Parts 9.5
In other contexts, Decode also means The memory-bandwidth-bound phase that generates output tokens one at a time after prefill..
Current numbers
Benchmark the selected media pipelineper-sample CPU decode cost of image/video vs text — the reason the GPU:CPU ratio must be set per modality
GTC 2026 (reported)Groq 3 LPU chip (~500 MB SRAM, ~150 TB/s) → LPX rack (256 LPUs, ~128 GB aggregate SRAM) fills the disaggregated decode/FFN slot (no HBM); Rubin GPUs retain prefill + attention. Rubin CPX reportedly pulled at GTC 2026 (no official NVIDIA cancellation)
xPyDruntime-reconfigurable disaggregation: x prefill workers feeding y decode workers, re-balanced live