Debugging Agent Failures

Agent bugs are strange. There's often no stack trace and no line of code that's obviously wrong. The agent just called the same tool eleven times, or picked the delete tool when it should have archived, or confidently told a customer their order shipped when the tool said it hadn't, or announced "Done!" with half the task untouched. The instinct is to add another sentence to the system prompt and hope. That works just often enough to become a habit, and just rarely enough to leave you with a prompt full of contradictory patches.

This lesson replaces hoping with a playbook. You'll learn a four-step debugging loop, how to find the first step where a run went wrong, and specific diagnoses and fixes for the four most common failures: loops, wrong tool use, hallucinated facts, and premature stops. You'll finish by locking in a fix with your eval set.

The debugging loop: reproduce, localize, fix, lock in#

Every agent bug, whatever the symptom, goes through the same four steps:

1. Reproduce. Get the exact input (and conversation history, if any) that caused the failure, and run it again, several times, since agents vary. A failure that happens 1 time in 10 needs a different approach than one that happens every time. Your traces give you the exact inputs; your eval harness can run them repeatedly with TRIALS=10.
2. Localize. Find the first step where things went wrong. Not the step where the failure became visible, but the earliest bad decision or bad data.
3. Fix the cause, not the symptom. Decide which layer the problem lives in (instructions, context, tools, the harness code, or genuinely beyond the model's ability) and make the smallest change at that layer.
4. Lock it in. Add the failing input to your eval set, confirm the fix across multiple trials, and run the full eval to make sure you didn't break something else.

Skipping step 2 is the most common mistake. If you fix the place where the failure appeared, you're usually patching a symptom.