AI agent glossary
Prompt caching
Reusing the already-processed beginning of a prompt across requests, which makes repeated calls with the same prefix faster and cheaper.
Agents send a lot of repeated text. Every turn of the loop resends the system prompt, the tool definitions, and the conversation so far. Prompt caching lets the provider store the processed version of that shared prefix so it does not need to be processed from scratch each time.
Cached input is typically billed at a much lower rate than fresh input and is returned faster. The exact discounts and how long a cache lasts vary by provider, so check the current pricing documentation.
To benefit, keep the stable parts of your prompt at the beginning and in the same order, and put content that changes, such as the latest user message, at the end. Small changes early in the prompt break the cache for everything after them.
Learn more in the course
Related terms
Go from definitions to doing
Learn to use and build AI agents in 65 practical lessons.