MTTI improves at 16k-GPU scale; MI355X gets faster; NVL72 cooling flows revised down

The share of Llama-3 training job interruptions due to network and configuration issues has decreased from approximately 10.7% to 10.1% as of January 1, indicating a potential improvement in topology management (Chapter 10.2). This change may enhance overall training reliability for AI workloads.

Mean Time to Interrupt (MTTI) for a 405B run on 16k H100 has improved from around 192 minutes to 186 minutes, suggesting that the frequency of checkpoints may be reduced, thus optimizing resource usage (Chapter 9.1).

In hardware advancements, the AMD MI355X FP4 has increased its performance from 9.2 PFLOPS to approximately 10.1 PFLOPS while maintaining the same memory capacity, which could influence decisions on hardware procurement (Chapter 7.1). Additionally, secondary-loop flow rates for the GB200 NVL72 racks have been adjusted, now ranging from 170 to 235 L/min, a significant reduction from previous figures, which may impact cooling strategies (Chapter 5.13).

If you're designing liquid loops now: the revised 170–235 L/min secondary-flow range for GB200 NVL72 racks changes CDU sizing headroom — check it against your basis of design before the piping package freezes. The MTTI improvement is good news for checkpoint cadence math (Ch 9.1), not a reason to relax it.