WebDataset
A shard-based training data format that packs many small samples into large sequential archives, so the data loader reads streams instead of hammering the file system's metadata plane.
Current numbers
< 50% vs > 90%GPU utilization: naive CSV/small-file loading vs sharded WebDataset/TFRecord + DALI; shard sweet spot ~100 MB–1 GB