Decode
In the data pipeline: turning stored, compressed samples (images, video, packed text shards) back into training-ready tensors — CPU-bound work that gates GPU utilization.
Context: Parts 9.5
In other contexts, Decode also means The memory-bandwidth-bound phase that generates output tokens one at a time after prefill..
Current numbers
LPX in full production, 2026-08-24Groq 3 LPX (256 LPUs, ~128 GB SRAM): NVIDIA announced full production 2026-08-24; decode tier beside Rubin GPUs; CPX pulled
About 180 GB resident; 40 ms memory lower bound. At most 9 sequences fit; 10 fail. Below about 3.2 TB/s the 50 ms memory budget fails.Fit one decode replica before buying more HBM: result and flip threshold
xPyDruntime-reconfigurable disaggregation: x prefill workers feeding y decode workers, re-balanced live