Latency

Latency is the delay between a request and its response. For a single model call it depends on the model, how much input it reads, and how much output it writes. For an agent, latency compounds, because a task might involve many model calls and tool calls in sequence.

Common ways to reduce it include using a smaller, faster model for simple steps, running independent tool calls in parallel, caching repeated prompt prefixes, keeping inputs short, and limiting how much reasoning the model does on easy tasks.

Perceived latency matters as much as actual latency. Streaming output and showing progress as the agent works make a long task feel far more responsive than a silent wait.