Architecture for Production Agents
Queues, durable execution, state, and sessions: the architecture that keeps long-running agents reliable.
On your laptop, an agent is a script: call the model, run a tool, loop until done, print the answer. In production, that same script meets reality. The load balancer kills requests after a minute, but your agent needs eight. A deploy restarts the server halfway through a run. A user closes the tab and comes back tomorrow expecting the result. A network blip causes a retry, and your agent creates the same support ticket twice.
None of these are model problems. They're architecture problems, and they have well-known solutions. This lesson walks through the architecture that keeps long-running agents reliable: job queues and background workers, durable execution with checkpoints, idempotent tools, and session state. You'll see working TypeScript for each piece and finish by deliberately crashing a worker mid-run to prove it recovers.
Why request-response breaks for agents
A typical web request takes milliseconds. An agent run takes anywhere from seconds to hours, and that changes everything:
- Timeouts. Load balancers, proxies, and serverless platforms cut off long requests. An agent run inside an HTTP handler will eventually be killed mid-step.
- Restarts. Deploys, crashes, scaling events, and out-of-memory kills all terminate processes. Any run living only in memory is lost.
- Disconnects. Users close laptops and lose connections. The work shouldn't depend on the browser staying open.
- Retries. Networks fail, APIs return overloaded errors, and clients retry. Without care, a retry repeats side effects: two emails, two charges, two tickets.
- Concurrency. Two messages arrive in the same conversation at once, and two agent loops start editing the same history.
The fix is to stop treating an agent run as a request and start treating it as a job: a unit of work with an ID, persisted state, and a lifecycle that survives any single process.