# Agent risk

> An agent that can act can also do harm, and a model can be talked into acting by text it reads. This topic teaches how to size what an agent can reach, grant it the least it needs, keep a human at the right decision points, and recognize the two classic attacks: prompt injection and data exfiltration.

Page: https://schubergphilis.github.io/ai-training/topics/safety/agent-risk/
License: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/

*Safety* · topic `safety/agent-risk`

An agent that can act can also do harm, and a model can be talked into acting by text it reads. This topic teaches how to size what an agent can reach, grant it the least it needs, keep a human at the right decision points, and recognize the two classic attacks: prompt injection and data exfiltration.

## Concepts

- **Blast radius**: Everything an agent's action can reach and change: the files it can write, the accounts it can act as, the systems its credentials open, the people its messages reach. Blast radius is decided by what the agent is connected to, not by what you asked it to do. Sizing it before a task is the first step in deciding how much autonomy to allow. [glossary](https://schubergphilis.github.io/ai-training/glossary/#blast-radius)
- **Permissions and least privilege**: Giving an agent only the access the current task needs: read where it needs to read, write only in the working folder, no network or credentials unless required. Least privilege limits what a mistake or an attack can do. Convenient broad access is the usual failure; a separate account, a sandbox or a scoped token is the usual fix. [glossary](https://schubergphilis.github.io/ai-training/glossary/#permissions-and-least-privilege)
- **Human in the loop**: A required human decision at a chosen point in an agent's work, such as before sending, paying, deleting or deploying. The human sees what the agent proposes and approves, edits or stops it. Placing the checkpoint at the irreversible step, rather than everywhere, keeps the agent useful while keeping accountability with a person. [glossary](https://schubergphilis.github.io/ai-training/glossary/#human-in-the-loop)
- **Prompt injection**: An attack where text the agent reads, such as a web page, email, document or code comment, contains instructions that the model follows as if they came from its user. The model cannot reliably tell data from commands. Any agent that reads untrusted content and can act is exposed, and the defense is limiting what it can do, not hoping it will refuse. [glossary](https://schubergphilis.github.io/ai-training/glossary/#prompt-injection)
- **Exfiltration**: Data leaving a boundary it should not cross, often as the second half of an injection: the agent is tricked into putting secrets, private files or conversation contents into a URL, a message or a public location. Any tool that can send data outward is an exfiltration path. Review those tools first and restrict where an agent may send anything. [glossary](https://schubergphilis.github.io/ai-training/glossary/#exfiltration)

## Links

- **Builds on:** [What an agent is](https://schubergphilis.github.io/ai-training/topics/concepts/what-is-an-agent/), [Verifying outputs](https://schubergphilis.github.io/ai-training/topics/safety/verification/)
- **Leads to:** [Quality with agents](https://schubergphilis.github.io/ai-training/topics/coding-with-agents/quality/), [Hooks, permissions, settings](https://schubergphilis.github.io/ai-training/topics/customizing-agents/hooks-permissions/), [Governance and oversight](https://schubergphilis.github.io/ai-training/topics/safety/governance/)
- **Competencies drawing on it:** [Verifies agent-written code before trusting it](https://schubergphilis.github.io/ai-training/competencies/coding-with-agents/verifies-agent-work/), [Connects an agent to tools and data safely](https://schubergphilis.github.io/ai-training/competencies/customizing-agents/connects-tools-safely/), [Judges the risk of letting an agent act](https://schubergphilis.github.io/ai-training/competencies/safety/judges-agent-risk/)

## Lessons

- [Why agent safety is different](https://schubergphilis.github.io/ai-training/safety/agent-risk/) (explanation)
- [Sizing an agent's blast radius](https://schubergphilis.github.io/ai-training/safety/sizing-the-blast-radius/) (tutorial)
- [Tracing a planted instruction to the tool that leaks](https://schubergphilis.github.io/ai-training/safety/tracing-a-planted-instruction/) (tutorial)

## Sources

- `AEC-10` [Guardrails and safety: why agent safety differs, defense layers, injection, human in the loop](https://lsimons.github.io/agent-engineer-course/10-guardrails-and-safety/), Agent Engineer Course (course)
- `Academy introduction-to-claude-cowork` [Introduction to Claude Cowork](https://academy.claude.com/courses/introduction-to-claude-cowork), Claude Academy (course)