When to Fine-Tune (and When Not To)
An honest guide to fine-tuning for agents: what it can and can't fix, the cheaper alternatives to try first, and how to evaluate the result.
Sooner or later someone on your team says it: "We should fine-tune a model on our data." Sometimes it's the right call. More often it's an expensive detour that solves a problem better handled by a clearer prompt, a retrieval step, or a better tool. It also quietly chains you to one model snapshot just as a better one comes out.
This lesson is an honest guide to that decision. You'll learn what fine-tuning actually changes, the cheaper alternatives to try first and the order to try them in, what fine-tuning is genuinely good at, the problems it can't fix, and the hidden costs that rarely make it into the proposal. If you do go ahead, you'll learn how to build the dataset and evaluate the result rigorously enough to know whether it worked.
What fine-tuning actually changes
Fine-tuning means continuing to train an existing model on your own examples, so the model's weights (its learned parameters) shift toward the behavior in those examples. Prompting changes what the model sees; fine-tuning changes what the model is.
The main variants you'll hear about:
| Variant | You provide | What it teaches |
|---|---|---|
| Supervised fine-tuning (SFT) | Example inputs with ideal outputs | "Respond like this" |
| Preference tuning (DPO is a common method) | Pairs of a better and a worse response | "Prefer this over that" |
| Reinforcement fine-tuning | Tasks plus a grader that scores attempts | "Get a higher score on this kind of task" |
| Distillation | Outputs from a large "teacher" model, used as SFT data for a smaller "student" | "Do what the big model does, cheaper" |
You'll also see LoRA (low-rank adaptation), a parameter-efficient technique that trains a small set of added weights instead of the whole model. It makes fine-tuning open-weight models affordable on modest hardware.
Where you can fine-tune matters as much as how. Some hosted providers offer fine-tuning for some of their models, and the options change often. Many of the most capable hosted models can't be fine-tuned by customers at all. Open-weight models (see open-weight and local models) can always be fine-tuned, using widely used open-source tooling such as Hugging Face's TRL library, but then you host and serve the result yourself. Check current provider docs before planning anything.