SLO · Service Level Objective
A target for a service metric such as latency or availability that an inference fleet is sized to meet.
Current numbers
1:1 vs 2:1–3:1Published scale-out examples: 1:1, 2:1–3:1 and a reported 7:1; derive yours from measured traffic, topology and SLO (contested)
31.6% combined-design savingSemiAnalysis Oct 2024 1,024-H100 training model: 31.6% combined VMS plus 2:1 saving; derive inference tiers from measured traffic, placement, failure headroom, and tail-latency SLO