DALI · Data Loading Library
NVIDIA's GPU-accelerated data-loading and preprocessing library, used with sharded formats and GPUDirect Storage to keep accelerators fed rather than stalled on input pipeline work.
Current numbers
< 50% vs > 90%GPU utilization: naive CSV/small-file loading vs sharded WebDataset/TFRecord + DALI; shard sweet spot ~100 MB–1 GB