FP16 · 16-bit floating point
A half-precision number format; it is the usual reference rung of the precision ladder, since each step down from it roughly doubles a chip's headline throughput.
Current numbers
~$1.90/M tokself-hosted Llama-3.1-70B FP16 on 8× H100 at $2.40/GPU-hr; ~2,800 tok/s aggregate (vLLM, batch 256, 512-in/512-out)