Fine-tuning

You assemble pairs of input and the output you wanted, and run a short training pass that adjusts the model's weights towards producing them. Most practical fine-tuning today is parameter-efficient — techniques such as LoRA train a small set of additional weights rather than the whole model, which is why it can cost tens of dollars instead of millions.

Quality of data dominates quantity. A few hundred carefully curated, consistent examples routinely beat tens of thousands of scraped ones, because the model learns whatever pattern is actually in the data, including your inconsistencies. For style and format work, a few hundred examples is a realistic target and a few thousand is usually the point of diminishing returns.

Reach for it when the behaviour you want is hard to describe but easy to demonstrate: a specific tone, a strict format, a classification boundary your team knows by feel. It also pays off when a prompt has grown so long that its cost on every call exceeds the one-off cost of training.

Before that, exhaust prompting and retrieval. They are faster to iterate on, cheaper to abandon, and they leave you free to switch models next quarter — a fine-tune ties you to the model you tuned.

Fine-tuning to add knowledge. This is the single most common misuse. Training on your documentation does not give the model reliable access to your documentation — it shifts its style towards it while leaving the facts as unreliable as ever, and it makes updating anything a retraining job. If the goal is "answer from our content", the answer is retrieval, not fine-tuning.

The second is skipping the eval set. Without a held-out set of real cases you cannot tell whether the tuned model is better or merely different, and a fine-tune that improves your favourite example while degrading everything else looks like success right up until release.