POLCA
Microsoft Research's power-oversubscription framework for LLM clusters (Patel et al., ASPLOS 2024). In the fleets it studied, inference left ~21% of provisioned power unused and synchronous training ~3%.
Current numbers
~3% training / ~21% inferenceUnused power headroom in POLCA's fleets (2024): training runs near peak and still swings, which a flat load bank misses
~30%more inference servers deployable under a fixed power budget via priority-based oversubscription (POLCA simulation)