How Agents Work
The mental model behind every agent
Understand what's happening under the hood: language models, the agent loop, tool calling, context windows, and how to pick the right model for a job.
- Lessons
- 6
- Reading time
- 55 min
- Level
- Beginner
- Free lessons
- 1
Lessons in this module
- 1.19 minFree
LLMs in Plain English
How large language models actually work — tokens, prediction, training, and reasoning — explained without math so you can predict their behavior.
- 1.29 min
The Agent Loop: Think, Act, Observe
The simple loop at the heart of every agent. Trace a real agent run step by step and learn where things go right and wrong.
- 1.311 min
Tools and Function Calling
How models call tools: schemas, tool calls, tool results, and why well-described tools make or break an agent.
- 1.49 min
Context Windows, Tokens, and Cost
What fits in an agent's working memory, how tokens translate to dollars, and how to keep long-running agents sharp and affordable.
- 1.58 min
Choosing the Right Model for the Job
Frontier vs. fast models, reasoning effort, cost/latency trade-offs, and a practical decision framework for picking a model per task.
- 1.69 min
Agents vs. Workflows: When to Use Which
Not everything should be an agent. Learn the spectrum from single prompts to chains to autonomous agents and how to choose.
About this module
Every AI agent, from a coding assistant in your terminal to a customer support bot, is built from the same few parts. A large language model reads text and predicts what comes next. Tools let it take actions, but the model never runs them itself: it writes a structured request, and ordinary code carries it out and returns the result. A simple loop repeats that cycle of thinking, acting, and observing until the task is finished.
Understanding these parts makes agent behavior predictable. Once you know that a model only sees what is in its context window, you can explain why an agent forgets an early decision in a long session. Once you know how tool calls work, you can see why a vague tool description leads to wrong actions. And once you know how tokens are counted, you can estimate what a run will cost before you start it.
This part covers the mental model behind every agent, without math. You will learn how LLMs work: tokens, next-token prediction, training, knowledge cutoffs, reasoning, and why models sometimes make things up. You will trace a real agent run turn by turn, follow a function call from request to result, and see what fills a context window and how that turns into cost.
The last two lessons are about judgment. One gives you a practical framework for choosing a model, weighing quality, cost, latency, and reasoning effort. The other explains the difference between AI agents and workflows, the spectrum from single prompts and prompt chains to fully autonomous agents, and how to decide how much autonomy a task actually needs.
Questions about How Agents Work
How do AI agents work?
An AI agent runs a language model in a loop. On each turn the model looks at the goal and everything so far, then either answers or asks to use a tool; the surrounding program runs the tool and feeds the result back. The loop ends when the model decides the task is done or a limit is reached.
What is function calling in LLMs?
Function calling, also called tool use, lets a model request an action by producing structured output that names a tool and its arguments, usually described with JSON Schema. Your code runs the function and sends the result back to the model. The model itself never executes anything.
What is a context window?
The context window is the maximum amount of text, measured in tokens, that a model can consider at once. It includes the system prompt, tool definitions, the conversation so far, and every tool result. As it fills, quality often degrades, and older material eventually has to be dropped or summarized.
What is the difference between an AI agent and a workflow?
In a workflow, your code decides the sequence of steps and the model fills in each one. In an agent, the model decides what to do next based on what it observes. Workflows are more predictable and usually cheaper; agents handle open-ended tasks where the steps can't be known in advance.
Why do LLMs hallucinate?
Models generate the most plausible continuation of text based on patterns learned in training, rather than looking facts up. When they lack the information, they can produce something that sounds right but isn't. Putting sources in the context and asking the model to cite them reduces this considerably.
Which LLM is best for building agents?
There is no single best model; the right choice depends on the task. A common approach is to start with the most capable model to establish what good output looks like, then try faster and cheaper models and keep the cheapest one that still meets your quality bar.