AI agent glossary
Model routing
Sending each request or step to the model best suited to it, such as a fast, cheap model for simple work and a more capable one for hard problems.
Not every step in an agent needs the most powerful model. Classifying an email, extracting a date, or summarizing a short page can often be done well by a smaller model that is faster and several times cheaper.
Model routing makes that choice deliberately. The routing can be fixed, where a given step always uses a given model, or dynamic, where a quick classifier or simple rules decide which model handles each request.
Routing only pays off if quality holds. Use evals to confirm the cheaper model is good enough for the steps you route to it, and keep a path to escalate to a stronger model when it is not.
A support system uses a small model to sort incoming tickets by topic and urgency, and passes only the complex, high-value ones to a larger model to draft replies.
Learn more in the course
Related terms
Go from definitions to doing
Learn to use and build AI agents in 65 practical lessons.