LoRA · Low-Rank Adaptation
A fine-tuning method that trains small low-rank adapter matrices instead of all weights, slashing cost and memory.
Current numbers
~0.1%share of parameters trained by a LoRA adapter vs full fine-tune (model-dependent)
A fine-tuning method that trains small low-rank adapter matrices instead of all weights, slashing cost and memory.