Quiz: AI Storage, Checkpointing & Data Infrastructure (Part 9)

10 questions from Part 9 of the guide. Score instantly; every explanation links to the chapter that derives the answer.

The primary sizing axis for AI training storage is…
IOPS onlyper-GPU bandwidth (GB/s/GPU) — capacity follows; starving GPUs of data is the failure modecapacity (PB) firstrack units consumed
The checkpoint-size rule of thumb the guide derives is roughly…
~14 bytes per parameter for BF16 weights + FP32 master weights + Adam state (recipe-dependent)~100 bytes per parameter~1 byte per parameterindependent of model size
The primary-store fork for training data is…
SQL vs NoSQLPOSIX parallel file systems (Lustre, GPFS, Weka, VAST) vs object-native pipelines — with the corpus format decision downstreamUSB drives vs tapelocal disks only
The media hierarchy for an AI cluster typically runs…
tape everywherenode-local NVMe scratch → networked all-flash → Ethernet-attached capacity/objectone tier fits allRAM only
On-disk corpus formats trade off as…
always use ZIPdatabases onlyall formats perform identicallysequential shards (WebDataset/MDS) for streaming throughput vs columnar (Parquet) for selective reads vs raw files for flexibility
The capacity-tier decision (on-prem vs cloud object) hinges significantly on…
compression ratios onlythe color of the consolelogo preferencewho owns the egress bill — repeated multi-PB reads across a cloud boundary can dominate storage economics
What evidence is required to quantify the benefit of prefix caching or KV offload?
Hit rate and prefix-length distribution, KV layout per rank/replica, model parallelism, tier capacity and transfer bandwidth/latency, concurrency, recompute cost, and the tail-latency SLOA universal users-per-GPU multiplierThe model parameter count aloneThe raw NVMe capacity in the server
For Llama 3 70B at 128K context, what does the guide's 40 GiB KV figure represent?
the per-rank HBM left after weights and runtimea universal spill threshold40 GiB aggregate KV state per sequence across participating ranks40 GiB on every tensor-parallel rank regardless of sharding
Data preparation should be run as…
a first-class, separately-sized cluster with its own CPU/storage/network profilea side job on the training clustersomething the storage vendor handlesa manual quarterly task
Checkpoint cadence trades off against…
model accuracynothing; checkpoint constantlygoodput — each save costs write-burst time, while longer intervals raise recompute loss (Meta's Llama 3 run on up to 16,384 H100s saw ~1 interruption every 3 h)network security

Check your understanding: Storage & Data