The Agent Threat Model
What can go wrong when software can act on its own — and the 'lethal trifecta' of risks to design around.
A chatbot that gets something wrong produces a bad paragraph. An agent that gets something wrong can delete a database, email your customer list to a stranger, or push broken code to production, all while sounding calm and helpful. The power that makes agents useful is the same thing that makes them dangerous: they turn words into actions.
Most agent incidents aren't caused by clever hackers. They come from ordinary design choices that quietly add up to a disaster, like giving one agent your inbox, the open web, and the ability to send email. This lesson gives you a threat model, a simple way to think about what can go wrong before it does. You'll learn the "lethal trifecta" that underlies most serious agent attacks, and you'll finish by auditing an agent you actually use.
Why agents change the security picture
Traditional software does exactly what its code says. If a function can't send email, no input will ever make it send email. Security work focuses on bugs: places where the code accidentally does something its authors didn't intend.
Agents break that assumption in three ways.
- Behavior is decided at runtime by a model. The code defines what the agent can do (its tools). The model decides what it will do, based on everything in its context window. Change the context and you change the behavior.
- Instructions and data travel in the same channel. A language model reads your system prompt, the user's request, a web page, and a tool result as one long stream of text. It has no hard boundary that says "this part is a command, this part is just content." That's the root of prompt injection, which gets its own lesson.
- Agents act with your authority. When an agent calls the GitHub API, it uses your token. When it sends an email, it comes from your address. Security people call this the confused deputy problem: a program with legitimate privileges gets tricked into using them on someone else's behalf.
Put those together and a new rule emerges: assume anything that can influence the agent's context can influence its actions. Everything else in this module follows from taking that rule seriously.