LoRA & QLoRA

Summary

Low-Rank Adaptation freezes the base model and trains small injected low-rank matrices instead; QLoRA adds quantization to fine-tune large models on consumer GPUs.

• 1Fine-tune a small open-weight model with LoRA using the Hugging Face `peft` library.
• 2Learn the key LoRA hyperparameters: rank, alpha, target modules.
• 3Try QLoRA on a larger model to see how quantization enables fine-tuning on limited hardware.