Fine-Tuning

Definition. Further training of a pretrained model on a task-specific dataset to adjust its weights so that it reliably produces a desired style, format or behaviour. Supervised fine-tuning trains on input-output pairs; preference tuning such as DPO trains on ranked pairs. It changes what the model tends to say, not what it knows, which is why it is a poor tool for injecting facts and a good tool for enforcing format, tone and narrow task competence.

How it works. Curate several hundred to several thousand high-quality examples in exactly the format you want at inference. Split into train, validation and held-out test sets. Choose a parameter-efficient method such as LoRA to keep cost and hardware modest. Train, then evaluate on the held-out set against the base model with the same prompt, and only deploy if the win is measurable and does not regress other behaviours.

Example. A firm needs every generated summary to follow a fixed four-section structure with specific headings and a maximum of 120 words. Prompting achieves 78 percent compliance. After fine-tuning a small model on 1,200 human-written examples, compliance reaches 99.2 percent and the average output length drops by 30 percent, cutting inference cost. The model is not smarter, it is simply more obedient to a format.

Use case. Fine-tune when you have a stable, well-defined task, a few hundred clean examples, and a business reason to use a smaller or cheaper model. It is also the right tool for domain-specific classification, structured extraction with unusual schemas, and reducing prompt length by baking instructions into weights.

Warnings. Fine-tuning on model-generated data amplifies the base model's errors and can collapse output diversity. Never fine-tune on data containing personal information without a lawful basis, and remember that a fine-tuned model can be made to memorise and regurgitate training examples.

Limitations. Fine-tuning cannot teach facts reliably, goes stale as knowledge changes, and requires re-running the whole pipeline for every update. It also tends to degrade general capability, a phenomenon called catastrophic forgetting, and it makes the model harder to audit because behaviour is encoded in weights rather than in a readable prompt.

Pros and cons
ProsCons
Can adapt a model to a stable task with clean training examples.Requires training and evaluation again as needs change.
← Browse all glossary terms