LoRA · Low-Rank Adaptation
A fine-tuning method that trains small low-rank adapter matrices instead of all weights, slashing cost and memory.
Current numbers
~0.003–0.022%Share of GPT-3 parameters trained in the original LoRA paper's two Table 4 configurations