less inter-site communication for DiLoCo (8 workers) vs fully-synchronous SGD, matching convergence on C4
~500xobserved
| Value kind | observed — Reported measurements, counts, and specifications keep the precision and scope stated by their source; an exact specification is not treated as a range. |
|---|---|
| As of | 2024 |
| Source | DiLoCo, Douillard et al. (arXiv:2311.08105) · Abstract: eight workers match fully synchronous optimization on C4 while communicating 500 times less |
| Review | checking…review by 2026-09-23 · standard cadence |
| Recorded changes | last 2026-06-29 |
| Claim id | less-inter-site-communication-for-diloco-8 |
Where the guide uses it
← Full numbers register — every date-stamped figure in the guide, with revision history.