assumed serialized-state recipe: 2B BF16 weights + 4B FP32 master + 4B+4B Adam moments = 14 bytes/param; ~7x the weight file
~14 bytes/paramderived
| Value kind | derived — Derived values are guide calculations; their result is only as fixed as the stated inputs, scope, and method. |
|---|---|
| Scope | An arithmetic recipe for the standard BF16-compute / FP32-master / Adam mixed-precision configuration, not a measured population value. VAST Data assumed 14 bytes/param to infer model sizes from checkpoint sizes (its own footnote puts the resulting uncertainty at about ±15%); the survey did not establish the recipe. 8-bit Adam moments give 8 bytes/param; optimizers that drop the second moment change it again. Verify the tensors the chosen framework actually serializes. |
| As of | 2026 |
| Source | VAST Data, 'Optimizing Checkpoint Bandwidth for LLM Training' — states 14 bytes/parameter as an assumption used to estimate parameter counts (footnote 1); the recipe itself is arithmetic |
| Review | checking…review by 2027-09-05 · standard cadence |
| Recorded changes | last 2026-09-04 · 2 revisions tracked |
| Claim id | checkpoint-state-per-parameter-2b-bf16-weights |
Where the guide uses it
- 9.1 Storage in the AI Lifecycle: Why It Determines GPU Efficiency
- 9.4 Checkpointing for Large-Scale Training
- 9.8 Sizing, Data Gravity & Resilience
- 13.9 Cluster-Scale Benchmarking, Reference Training & Storage/Scheduler Validation
← Full numbers register — every date-stamped figure in the guide, with revision history.