AI agent glossary
Model gateway
A shared service that sits between an organization's agents and model providers, handling keys, access rules, logging, and cost tracking in one place.
Without a gateway, every team gets its own API keys, logs usage its own way, and nobody can answer what the company spends on models or what data was sent where. A gateway gives all agents one internal endpoint that forwards requests to approved providers.
Because every call passes through it, the gateway is the natural place for shared controls: which teams may use which models, spending limits, request logging and tracing, redaction of sensitive data, and fallbacks when a provider is down.
It is usually the first component of an internal agent platform. Keep it thin and fast, and design it as a paved road teams want to use rather than a gate they try to route around.
Every internal agent calls the company gateway with a team token. The gateway checks the team's model allowlist and budget, forwards the call, and logs tokens and cost per team.
Learn more in the course
Related terms
Go from definitions to doing
Learn to use and build AI agents in 130 practical lessons.