Scale-out
Connecting many nodes into a looser cluster fabric (InfiniBand or Ethernet) to scale beyond one coherent domain.
Current numbers
~1.8 TB/s vs ~400 Gb/sscale-up (NVLink5/GPU) vs scale-out (per-NIC) bandwidth — roughly 18x apart per direction (900 vs 50 GB/s each way)
~18xNVLink5 per-GPU BW (1.8 TB/s bidirectional, ~900 GB/s/dir) vs ~400G scale-out NIC (~50 GB/s/dir) — keep collectives in scale-up
1.8 TB/sNVLink-5 per-GPU scale-up bandwidth, bidirectional (NVL72 = ~130 TB/s rack aggregate); ~18x a 400 Gb/s scale-out NIC per direction (900 vs 50 GB/s each way)