Skip to content

Guardrails

Adrià Castany 3 min

What is guardrails?

AI guardrails are the controls that stop an AI system doing or saying certain things. They set which sources it may take data from, which actions it may run, which topics it does not touch and what needs a person to confirm. They are applied outside the model rather than by trusting it to obey.

Also called

AI guardrails, LLM guardrails, safety limits

Topic
AI in support
On this page
  1. Asking for it in the prompt is not a guardrail
  2. Why it matters
  3. Input, output and action guardrails
  4. The three levels
  5. Examples in customer support
  6. How to test them
  7. Separate reading from writing
  8. Irreversible things go through a person
  9. How Intake handles it
  10. Frequently asked questions

Asking for it in the prompt is not a guardrail

Writing "do not cancel subscriptions" in the instructions is a request, and a model can ignore it given a strange enough conversation.

The guardrail is that the cancel tool is not available, or that it requires human confirmation. The first is an intention; the second is a limit.

Why it matters

Because an agent connected to your API can do things with consequences, and the difference between an incident and a scare is whether the limit was in the code or in a sentence.

It is also the first thing the security team of any mid-sized customer asks about.

Input, output and action guardrails

They are classified by the moment they act.

A chatbot that only answers needs the first two. An agent connected to the company’s systems needs all three.

  • Input. They check what reaches the model, such as an off-topic request, a message trying to override the instructions (prompt injection) or personal data that should not be passed on.
  • Output. They check the answer before anyone sees it, looking for a fact that is not in the sources, a promise the company cannot keep or content that breaks policy.
  • Action. They decide which tools the system can call at all and which ones wait for a person to confirm.

The three levels

What it can read, what it can write and what it can say.

The first is controlled with permissions, the second with which tools you expose plus confirmation on anything irreversible, and the third by reviewing outputs.

Examples in customer support

The ones almost any company needs from day one.

  • The agent answers only from the company’s own content and says so when the answer is not there.
  • It can look up an invoice but cannot issue a refund.
  • It never quotes a price or a deadline that is not written in a source.
  • Legal threats, complaints about a person and cancellations go straight to the team.
  • Data from one customer never appears in a conversation with another.

How to test them

Write the conversations that should trip each guardrail and run them before every change to the agent.

A refund request outside the policy, a message that says "ignore your instructions", a question the documentation does not answer. A guardrail nobody has tested is an assumption. After launch, read the conversations where the agent refused or handed off, because that is where you see whether the limits sit in the right place.

Separate reading from writing

It is the line that pays off most.

Looking up a plan breaks nothing; changing it does. Starting by exposing reads only resolves a good share of conversations at no risk, and leaves writes for when there is confidence.

Irreversible things go through a person

Canceling, deleting, refunding.

Even if the agent is wrong no more often than a person, when it is wrong there will be nobody watching to notice in time.

How Intake handles it

Frequently asked questions

What are AI guardrails?

Controls that limit what an AI system can read, do and say. They sit around the model, in permissions, in the tools it is given and in checks on its answers, so they hold even when the model gets something wrong.

What are some examples of AI guardrails?

Answering only from approved sources, refusing topics outside the product, blocking attempts to override the instructions, keeping one customer’s data out of another’s conversation, and requiring a person to approve refunds, cancellations or deletions.

Are guardrails the same as content moderation?

Moderation is one kind, an output check for harmful or offensive content. Guardrails also cover what the system can read, which actions it can take and which topics are out of scope, and for an agent connected to your systems those matter more.

Do guardrails go in the prompt?

Instructions go there, and they help. The guardrail itself goes in the system. If an action must not happen, the tool is not exposed or it demands confirmation. Trusting the model to obey is not a limit.

Which actions should always require confirmation?

The irreversible ones and the ones that touch money. Cancellations, deletions, refunds and downgrades.

Glossary

Keep reading

Glossary

Related terms

See the full glossary

Add it without dropping what you use

15 minutes, no commitment