Quiz: AI Storage, Checkpointing & Data Infrastructure (Part 9)

10 questions from Part 9 of the guide. Score instantly; every explanation links to the chapter that derives the answer.

The primary sizing axis for AI training storage is…
capacity (PB) firstper-GPU bandwidth (GB/s/GPU) — capacity follows; starving GPUs of data is the failure modeIOPS onlyrack units consumed
The checkpoint-size rule of thumb the guide derives is roughly…
~1 byte per parameter~14 bytes per parameter (weights + optimizer state + metadata)~100 bytes per parameterindependent of model size
The primary-store fork for training data is…
USB drives vs tapePOSIX parallel file systems (Lustre, GPFS, Weka, VAST) vs object-native pipelines — with the corpus format decision downstreamSQL vs NoSQLlocal disks only
The media hierarchy for an AI cluster typically runs…
one tier fits allnode-local NVMe scratch → networked all-flash → Ethernet-attached capacity/objectRAM onlytape everywhere
On-disk corpus formats trade off as…
all formats perform identicallysequential shards (WebDataset/MDS) for streaming throughput vs columnar (Parquet) for selective reads vs raw files for flexibilityalways use ZIPdatabases only
The capacity-tier decision (on-prem vs cloud object) hinges significantly on…
logo preferencewho owns the egress bill — repeated multi-PB reads across a cloud boundary can dominate storage economicsthe color of the consolecompression ratios only
The KV-cache decision for an inference fleet is whether to treat it as…
always ephemeralephemeral HBM state (recompute on miss) vs a managed multi-tier store (HBM → DRAM → NVMe) — prefix caching + offload serves ~10× more users per GPUa security risk to disablepart of the model weights
One Llama 3 70B request at 128K context holds roughly how much KV state?
~1 GB~42 GB~500 GB~4 TB
Data preparation should be run as…
a side job on the training clustera first-class, separately-sized cluster with its own CPU/storage/network profilea manual quarterly tasksomething the storage vendor handles
Checkpoint cadence trades off against…
nothing; checkpoint constantlygoodput — each save costs write-burst time, but longer intervals raise the recompute loss when failures land (~every 3 h at 16k-GPU scale)model accuracynetwork security

Check your understanding: Storage & Data