Fine-tuning and RAG
Fine-tuning teaches the model how to answer.
RAG hands it, on every question, the content to answer with. If the documentation changes, with RAG you update the document, and with fine-tuning you would have to train again. For a customer service agent to know the product, RAG is what gets used.
Why it matters
Many people assume that putting AI into support means training it on the company’s data.
It almost never does, and believing so delays projects and raises legal doubts that do not apply. Knowing what fine-tuning solves and what it does not lets you ask a vendor the right question. Where the agent looks for the answer and whether conversations are used to train anything.
Fine-tuning versus RAG
They are two ways of adapting a model, and they solve different problems.
| Fine-tuning | RAG | |
|---|---|---|
| What changes | The model itself | What the model reads when answering |
| Good for | Style, format and repeated tasks | Knowledge that changes |
| When content changes | Train again | Update the document |
| Cites the source | No | Yes, the passage used |
| Data needed | Many well-chosen examples | The existing documentation |
When fine-tuning makes sense
Fine-tuning pays off in three cases.
- A narrow, repeated task, such as classifying conversations by topic.
- A very strict output format the model does not respect from instructions.
- Very high volume, where a small tuned model works out cheaper.
When it does not
It is no use for making the model know prices, policies or the state of an account.
That information changes and has to be looked up at the time. It does not prevent hallucinations either. A tuned model answers in the company’s tone even when it makes the fact up.
The questions about data
Before fine-tuning you need to know which conversations are used as examples, whether they contain personal data and who keeps the resulting model.
With RAG those questions are simpler, because the model is not modified.
