LoRA fine-tuning
-
LoRA vs Full Fine-Tuning GPU Memory for Enterprise Models
Full fine-tuning stores gradients and optimizer state for every parameter. LoRA trains small adapter
- 1
Full fine-tuning stores gradients and optimizer state for every parameter. LoRA trains small adapter