LLM · Large Language Model
An autoregressive transformer trained on text at scale; its training and inference shapes drive the facility designs in this guide.
Current numbers
~1–3% tuned servingBlackwell (B200) confidential-computing overhead for a tuned LLM serving stack; 30–40% for a stock stack, ~10–13% for eight-GPU training
30%peak grid-demand cut while training Megatron-LLM, measured on a GB200 rack fitted with the GB300 power shelf
~43.4%Failure rate, top 5% most resource-intensive LLM tasks (Alibaba Unicron, Dec 2023); ~73% of failures fixable by restarting the job