# Guardrails and production

> Running an agent for real means surrounding it with controls and treating its changes like any other risky deployment. This topic covers layered defenses, gating deploys on evaluation results, rolling changes out gradually, managing cost, operating the model API itself, the security risks specific to agents, and the protocols agents use to talk to tools and to each other.

Page: https://schubergphilis.github.io/ai-training/topics/building-agents/production/
License: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/

*Building agents* · topic `building-agents/production`

Running an agent for real means surrounding it with controls and treating its changes like any other risky deployment. This topic covers layered defenses, gating deploys on evaluation results, rolling changes out gradually, managing cost, operating the model API itself, the security risks specific to agents, and the protocols agents use to talk to tools and to each other.

## Concepts

- **Defense layers**: No single control stops every bad outcome, so production agents stack several: policy in the system prompt, input and output filtering, permission limits on tools, human approval at irreversible steps, monitoring and alerting, and the ability to stop the agent. Each layer catches what the previous ones miss, and the design assumes the model itself will sometimes be fooled. [glossary](https://schubergphilis.github.io/ai-training/glossary/#defense-layers)
- **Eval-gated deploys**: Treating any change to prompt, tools, model or harness as a release that must pass the evaluation suite before it reaches users, the same way code must pass tests. The gate turns evaluation from an occasional study into a continuous control and catches the regression that a small wording change or a model update introduces. [glossary](https://schubergphilis.github.io/ai-training/glossary/#eval-gated-deploys)
- **Rollout strategies**: Releasing a change to a fraction of traffic first, comparing its metrics with the current version, and widening or reverting based on what is observed. Shadow runs, canaries and gradual percentages all apply to agents. Because model behavior is non-deterministic and evaluation is imperfect, gradual rollout is the check that evaluation cannot give. [glossary](https://schubergphilis.github.io/ai-training/glossary/#rollout-strategies)
- **Cost**: What an agent spends per task in tokens, tool calls and time, and how that scales with users. Agents multiply model cost by their number of turns, and a runaway loop or verbose tool can multiply it again. Track cost per run, set budgets that stop a run, cache what repeats, and route simple steps to smaller models. [glossary](https://schubergphilis.github.io/ai-training/glossary/#cost)
- **Provider operations**: The mechanics of calling a model provider reliably. Pin an exact model ID, one that names a single fixed version, so the model changes only when you choose. Use prompt caching so the stable prefix of a system prompt and tool list is processed once and read back at a lower price on later calls. Retry retriable errors, such as rate limits and overload, with a backoff. Report terminal ones, such as an invalid request or a content refusal, to the caller. Set timeouts on every call. [glossary](https://schubergphilis.github.io/ai-training/glossary/#provider-operations)
- **Agent-specific security risks**: Risks that appear when a model can act: prompt injection through content the agent reads, exfiltration through tools that send data outward, over-permissioned tools that turn a wrong decision into real damage, and confused-deputy situations where the agent spends its own authority for whoever wrote the text it acts on. They are mitigated by limiting capability and reviewing actions, not by better prompts. [glossary](https://schubergphilis.github.io/ai-training/glossary/#agent-specific-security-risks)
- **Protocols (MCP, A2A)**: Standards for how agents connect to things. The Model Context Protocol standardizes how an agent reaches tools and data through servers. Agent-to-agent protocols standardize how one agent discovers, delegates to and exchanges results with another across organizations. Shared protocols reduce integration work and also widen the attack surface, so each connection is a trust decision. [glossary](https://schubergphilis.github.io/ai-training/glossary/#protocols-mcp-a2a)
- **Trace-driven evaluation**: The cycle in which the traces of a running agent feed its evaluation. Real use is logged, people read the traces, failures found there become new evaluation cases, and the next change is tested against them. Each round brings the evaluation set closer to what users actually do, so each improvement is measured on problems that users really have. It needs logging from the start and a habit of reading the traces themselves as well as the scores. [glossary](https://schubergphilis.github.io/ai-training/glossary/#trace-driven-evaluation)

## Links

- **Builds on:** [Evaluation and testing](https://schubergphilis.github.io/ai-training/topics/building-agents/evaluation/), [Governance and oversight](https://schubergphilis.github.io/ai-training/topics/safety/governance/), [Connecting tools with MCP](https://schubergphilis.github.io/ai-training/topics/customizing-agents/mcp/)
- **Leads to:** nothing yet
- **Competencies drawing on it:** [Evaluates an agent's quality systematically](https://schubergphilis.github.io/ai-training/competencies/building-agents/evaluates-agents/), [Runs an agent in production with guardrails](https://schubergphilis.github.io/ai-training/competencies/building-agents/runs-in-production/)

## Lessons

- [How a running agent gets attacked](https://schubergphilis.github.io/ai-training/building-agents/agent-attack-surface/) (explanation)
- [Protocols for tools and for other agents](https://schubergphilis.github.io/ai-training/building-agents/agent-protocols/) (explanation)
- [Cost, caching and pinning the model](https://schubergphilis.github.io/ai-training/building-agents/cost-and-provider-operations/) (tutorial)
- [Gating a deploy on the evaluation](https://schubergphilis.github.io/ai-training/building-agents/eval-gated-deploys/) (tutorial)
- [Growing the evaluation set from traces](https://schubergphilis.github.io/ai-training/building-agents/evals-from-traces/) (tutorial)
- [Guardrails in layers](https://schubergphilis.github.io/ai-training/building-agents/guardrails-in-layers/) (tutorial)

## Sources

- `AEC-10` [Guardrails and safety: why agent safety differs, defense layers, injection, human in the loop](https://lsimons.github.io/agent-engineer-course/10-guardrails-and-safety/), Agent Engineer Course (course)
- `AEC-11` [From prototype to production: eval-gated deploys, rollout, cost](https://lsimons.github.io/agent-engineer-course/11-from-prototype-to-production/), Agent Engineer Course (course)
- `AEC-14` [Agent protocols, MCP and A2A](https://lsimons.github.io/agent-engineer-course/14-agent-protocols-mcp-and-a2a/), Agent Engineer Course (course)
- `Brilliant SEC` Security and adversarial thinking, Brilliant, Coding with AI skills map (reference)
- `Academy model-context-protocol-advanced-topics` [Model Context Protocol: Advanced topics](https://academy.claude.com/courses/model-context-protocol-advanced-topics), Claude Academy (course)
- `Academy claude-platform-101` [Claude Platform 101](https://academy.claude.com/courses/claude-platform-101), Claude Academy (course)
- `Hardy 1988` [The Confused Deputy (or why capabilities might have been invented)](https://doi.org/10.1145/54289.871709), ACM SIGOPS Operating Systems Review 22(4) (1988), 36-38 (paper)
- `Husain 2024` [Your AI Product Needs Evals](https://hamel.dev/blog/posts/evals/), hamel.dev (reference)
- `OpenAI Academy optimize-ai-application-performance-1k50h` [Optimize AI Application Performance](https://academy.openai.com/public/courses/optimize-ai-application-performance-1k50h), OpenAI Academy (course)