Evaluator–Optimizer Loops
Pair a generator with a critic that scores and sends work back: when this loop improves quality, how to write the critic, and when to stop.
Ask a model for a first draft and you usually get something decent. Get a sharp editor to mark it up, hand the notes back, and the second draft is often much better. An evaluator–optimizer loop automates that exchange. One agent, the generator, produces work. A second agent, the evaluator (or critic), checks it against explicit criteria and sends it back with specific fixes. The loop repeats until the work passes or stops getting better.
Built well, this is one of the most reliable ways to raise output quality. Built badly, it burns tokens on a critic that approves everything, or on one that never stops nitpicking. In this lesson you'll learn when the loop pays off, how to write a critic whose feedback a generator can act on, and how to decide when to stop. You'll also build a working loop in TypeScript.
What the loop looks like
The shape is simple:
task ──► GENERATOR ──► draft ──► EVALUATOR ──► pass? ──yes──► done
▲ │
└──── specific fixes ◄──┘ no (and still improving)It's worth separating this from three neighbors you've met or will meet:
- Reflection, which you built as a minimal write, review, rewrite loop in Agentic Design Patterns. This lesson turns that sketch into a production pattern: a critic on a separate model, layered code checks, several stopping rules, and calibration.
- The reviewer stage in Handoffs and Shared State reports problems once and stops. An evaluator–optimizer loop closes the circle: the findings go back to the generator.
- LLM-as-judge in the evals module scores outputs offline to measure a system. Here the judge runs inside the system, at runtime, and its verdict changes what ships.
What makes it multi-agent is context separation. The evaluator starts fresh each round. It sees the task, the criteria, and the draft, and nothing else. It never sees the generator's reasoning, so it can't be persuaded by it. That's the "independent verification" benefit from When (and When Not) to Go Multi-Agent, put to work.