Short answer
Guardrails are the controls that constrain what an AI system may say and do: topic limits, validation of output against an expected structure, blocked actions, spending caps, and filters on both the incoming request and the generated response. Guardrails are enforced in the surrounding code, because an instruction inside a prompt can be argued with.
Also called: AI guardrails, Output validation, AI safety controls
The layers sit in different places for a reason. Input checks catch requests that are out of scope or that attempt to override instructions. Output checks confirm a response parses, stays on topic, and carries no customer data it should not. Action-level permissions are the strongest layer, because a system with no write access to the payments table cannot be talked into issuing a refund, whatever a cleverly worded message tells it. Prompt injection, where instructions hidden in a document or web page are read by the model as commands, is why permissions cannot be replaced by careful wording.
There is a real trade-off to manage. Every restriction removes some capability, and a system tightened until it refuses anything unusual gets abandoned by the people it was built for. The workable setting is generally strict about actions and money, permissive about drafting and suggestion, with everything logged. Logs are what turn a vague complaint that the AI got it wrong into a specific input, output and decision that can be reproduced and fixed.
Common questions
What is prompt injection?
An attack where instructions are hidden inside content the model reads, such as a web page, an email or an uploaded document, and the model follows them as though they came from you. It is not a solved problem. The dependable defence is limiting what the system is permitted to do, so a successful injection cannot reach anything valuable.
Where this comes up in our work
Related terms
AI Agent
An AI agent is a system that uses a language model to decide which actions to take, then takes them through connected tools: reading a database, sending an email, updating a booking record.
Human-in-the-Loop
Human-in-the-loop is a design pattern where an AI system prepares work and a person approves, edits or rejects it before it takes effect: drafted replies queued for a one-click send, extracted invoice fields shown for confirmation.
Hallucination (AI)
A hallucination is output that is fluent, plausible and false: an invented citation, a policy the company never wrote, a confidently wrong figure.
Prompt Engineering
Prompt engineering is the practice of writing model instructions that produce the required output reliably: stating the role, the task, the constraints, the output format, and one or two worked examples.
Reading definitions because you are scoping a project? Skip ahead and just ask.