Guardrails
Guardrails are the restrictions that stop an AI system doing or saying certain things: which sources it may take data from, which actions it may run, which topics it does not touch and what requires confirmation from a person. They are applied outside the model rather than by trusting it to obey.
What is guardrails?
Guardrails are the restrictions that stop an AI system doing or saying certain things: which sources it may take data from, which actions it may run, which topics it does not touch and what requires confirmation from a person. They are applied outside the model rather than by trusting it to obey.
Also: safety limits · constraints
Asking for it in the prompt is not a guardrail
Writing "do not cancel subscriptions" in the instructions is a request, and a model can ignore it given a strange enough conversation. The real guardrail is that the cancel tool is not available, or that it requires human confirmation. The first is an intention; the second is a limit.
Why it matters
What changes in a SaaS
Because an agent connected to your API can do things with consequences, and the difference between an incident and a scare is whether the limit was in the code or in a sentence. It is also the first thing any mid-sized customer security team asks about.
Guardrails in detail
The three levels
What it can read, what it can write and what it can say. The first is controlled with permissions, the second with which tools you expose plus confirmation on anything irreversible, and the third by reviewing outputs.
Separate reading from writing
It is the line that pays off most. Looking up a plan breaks nothing; changing it does. Starting by exposing reads only resolves a good share of conversations at no risk, and leaves writes for when there is confidence.
Irreversible things go through a person
Cancelling, deleting, refunding. Not because the agent will be wrong more often than a person, but because when it is wrong there will be nobody noticing in time.
How Intake handles it
Questions about guardrails
Do guardrails go in the prompt?
Instructions go there, and they help. The real guardrail goes in the system: if an action must not happen, the tool is not exposed or it demands confirmation. Trusting the model to obey is not a limit.
Which actions should always require confirmation?
The irreversible ones and the ones that touch money: cancellations, deletions, refunds and downgrades.
Related terms
A term on its own is only half understood. These come up in the same conversation.
Tool calling
Tool calling is the mechanism by which a language model can run functions you define instead of only writing. You describe the available tools, the model decides which one to use and with what arguments, and it receives the result to fold into its answer.
System prompt
The system prompt is the fixed set of instructions a model receives before every conversation: who it is, what it can and cannot do, what tone it answers in and what to do when it does not know something. The user never sees it and it is sent on every request.
AI agent
An AI agent is a system that, beyond generating text, reads data and takes actions to finish a task. In support that means it checks the actual state of the customer account before answering, and when the answer involves doing something, it does it instead of explaining how.
Evals — Evaluation
An evaluation is a set of test cases used to check, repeatably, whether an AI system answers as it should. Each case carries an input and a success criterion, and the whole set is re-run every time something changes: the prompt, the model or the documentation.
Hallucination
A hallucination is an answer from a language model that sounds right and is not: an invented figure, a feature that does not exist, a plausible number nobody checked. It is not a bug, it is a direct consequence of how a model works — it produces the most probable continuation, not the verified one.
Context window
The context window is everything a language model can have in front of it at once to produce an answer: the system instruction, the conversation history, the passages retrieved from documentation and the data looked up. It is measured in tokens and it has a ceiling.
