Capstone 2: A Customer Support Agent

Customer support is where many teams put their first agent in front of real people, and it's where agents fail most publicly. A support agent that invents a refund policy, leaks another customer's invoice, or argues with someone reporting a hacked account does real damage in a single reply.

In this capstone you'll build a support agent for a fictional app that does the job properly. It answers from a knowledge base instead of from memory, looks up only the signed-in customer's account, issues refunds only within limits enforced in code, and hands off to a human when the rules say so. Then you'll write the part most tutorials skip: an eval suite that tells you, in a couple of minutes, whether your latest prompt tweak made things better or worse.

What you'll build#

A support agent for Acme Planner, a made-up project-planning app, with:

• Knowledge base retrieval: a search_help_center tool over help articles.
• Account tools: the customer's plan, status, and invoices, plus a guarded refund_invoice tool.
• Escalation: an escalate_to_human tool, rules in the system prompt, and a code-level safety net for topics that must always reach a person.
• A chat CLI so you can talk to it as a customer.
• An eval suite of realistic cases graded by code checks and an LLM judge.

Billing data lives behind a small interface with a mock implementation, so everything runs with only an Anthropic API key. When you connect your real billing system, you implement one class.