less inter-site data exchanged by DiLoCo-class intermittent sync vs full synchronous DP — what makes cross-region training viable
100–500xobserved
| Value kind | observed — Reported measurements, counts, and specifications keep the precision and scope stated by their source; an exact specification is not treated as a range. |
|---|---|
| As of | 2025 |
| Source | Google DeepMind · Abstract; eight-worker DiLoCo matches fully synchronous optimization on C4 while communicating 500 times less |
| Review | checking…review by 2026-10-01 · standard cadence |
| Recorded changes | last 2026-08-24 · 2 revisions tracked |
| Claim id | less-inter-site-data-exchanged-by-diloco-class |
Where the guide uses it
Not quoted in a chapter yet; it is kept in the curated register.
← Full numbers register — every date-stamped figure in the guide, with revision history.