Storage & Data

Storage decides whether a $40,000 accelerator computes or idles, and getting it right means designing for four mutually-hostile I/O personalities that compete for the same flash.

A parallel file system keeps a $30-40k GPU saturated only if its metadata plane survives billions of small files; sequential bandwidth is the easy part every all-flash platform already clears.

Whether a $40k GPU stays fed is set by per-request latency more than aggregate bandwidth, which is why the 2026 data path routes bytes from flash straight into HBM and takes the host CPU out of the loop.

Checkpointing is a training cluster's goodput control knob — the interval, tier, and write bandwidth you choose decide how many GPU-hours each failure erases, and at frontier scale failures arrive constantly.

The data loader sits in the critical loop of every training step; get its format, sharding, and CPU budget wrong and the GPUs stall, burning depreciation while they wait to be fed.

Object storage holds the whole corpus, every checkpoint lineage, and every shipped model; its cost turns less on vendor choice than on whether you build it as a flash-fronted serving layer the GPUs read from.

Inference made the KV-cache a first-class storage problem: its bytes now spill past HBM, and where you let them land — DRAM, CXL, NVMe, or Ethernet-flash — sets your tokens-per-second and cost-per-token.

Size the hot tier by per-GPU bandwidth, place the corpus where the compute can reach it, and isolate the checkpoint incast — miss any of the three and idle accelerators absorb the cost.

Before any GPU sees a token, a second cluster must dedupe, filter, decontaminate, and tokenize trillions of tokens — undersize it and you starve the training fleet or waste GPU-hours on string processing.