Fine-Tuning

A pretrained model already knows a lot. Fine-tuning adapts it with your own examples, a form of transfer learning: show input-output pairs, compute the loss, and update with gradient descent, usually with a small learning rate so it does not forget what it knew.

Updating every weight of a billion-parameter model is expensive, and each fine-tuned copy is as big as the original. LoRA (low-rank adaptation) avoids both:

1. Freeze the original weights.
2. Next to selected weight matrices, add two thin matrices, B and A, whose product is a correction to that weight.
3. Train only B and A.

Because they are thin (the “rank” might be 8 or 16), they hold a tiny fraction of the parameters, often well under 1%. The resulting adapter is a small file you can swap in and out. QLoRA goes further by keeping the frozen base model quantized to 4 bits.

When to choose what: fine-tuning shapes behavior (format, tone, a narrow skill); RAG supplies knowledge that changes or must be cited; good prompting is the cheapest first try.

The math

For a frozen weight $W \in \mathbb{R}^{d \times k}$, LoRA learns $B \in \mathbb{R}^{d \times r}$ and $A \in \mathbb{R}^{r \times k}$ with $r \ll \min(d, k)$:

$$h = W x + B A x$$

Trainable parameters drop from $dk$ to $r(d + k)$.