SGLang
An open-source LLM serving framework with fast structured generation and aggressive KV-cache reuse.
Current numbers
~40%MI355X (ROCm/SGLang FP8) cost-per-Mtoken advantage vs B200 (SGLang) at points on the GLM-5 single-node frontier — gap reversed on funded workloads