Evals — Evaluation
An evaluation is a set of test cases used to check, repeatably, whether an AI system answers as it should. Each case carries an input and a success criterion, and the whole set is re-run every time something changes: the prompt, the model or the documentation.
What is evaluation?
An evaluation is a set of test cases used to check, repeatably, whether an AI system answers as it should. Each case carries an input and a success criterion, and the whole set is re-run every time something changes: the prompt, the model or the documentation.
Also: evals · evaluations
Testing by hand is not evaluating
Typing four questions into the chat after a change and seeing whether it answers well is a check, not an evaluation. What makes it an evaluation is that it is always the same set, so results can be compared: without that you cannot tell whether the change improved anything or just moved the problem.
Why it matters
What changes in a SaaS
Because in an AI system any change can make worse things that already worked, and you cannot see that by looking. Adjusting the prompt to fix one answer and breaking ten others is normal without evaluations, and the worst part is that nobody finds out until a customer says so.
Evaluation in detail
Where the cases come from
From your own conversations. The fifty most frequent questions, plus the cases that went wrong, plus the limits you want to protect — what it must not answer. Inventing generic cases gives you an evaluation that always passes and protects nothing.
What gets checked
That the answer contains the right fact, that it does not contain what it must not, and that it escalates when it should. The third is the most forgotten and the most expensive when it fails.
When they are run
Before every change that reaches customers. It is the same discipline as tests before a deploy, applied to something non-deterministic — which is why it needs it more.
Questions about evaluation
How many cases does an evaluation need?
Start with thirty or forty taken from your real conversations. That is enough to catch a regression and few enough that somebody will maintain them.
Can answers be evaluated automatically?
Partly. Checking that a specific fact appears, or that something forbidden does not, automates well. Judging whether an answer is well written still needs judgement, even when another model is used to score it.
Related terms
A term on its own is only half understood. These come up in the same conversation.
Guardrails
Guardrails are the restrictions that stop an AI system doing or saying certain things: which sources it may take data from, which actions it may run, which topics it does not touch and what requires confirmation from a person. They are applied outside the model rather than by trusting it to obey.
System prompt
The system prompt is the fixed set of instructions a model receives before every conversation: who it is, what it can and cannot do, what tone it answers in and what to do when it does not know something. The user never sees it and it is sent on every request.
Hallucination
A hallucination is an answer from a language model that sounds right and is not: an invented figure, a feature that does not exist, a plausible number nobody checked. It is not a bug, it is a direct consequence of how a model works — it produces the most probable continuation, not the verified one.
AI agent
An AI agent is a system that, beyond generating text, reads data and takes actions to finish a task. In support that means it checks the actual state of the customer account before answering, and when the answer involves doing something, it does it instead of explaining how.
RAG — Retrieval-augmented generation
Retrieval-augmented generation, or RAG, is the technique of finding relevant information in your own sources and handing it to a language model so it writes the answer with that. The model does not memorise the content: it looks it up at answer time.
Latency
Latency is how long an AI system takes to return an answer from the moment it receives the question. It includes the information retrieval, the lookups against other systems and the text generation, and it is perceived whole even though the model is only one part of it.
