Max tokens

When you call a model API you set a ceiling on the length of the reply. In the Anthropic Messages API this is the required max_tokens parameter. The model stops when it reaches that ceiling, even mid-sentence, and the response's stop reason says so.

Max tokens is often confused with the context window. The context window is the total the model can consider at once, including your prompt, the conversation history, and the reply. Max tokens only bounds the reply. Each model also has its own maximum output length, so check the documentation for the model you use.

Set it high enough for the longest legitimate answer, including long tool inputs such as a full file being written, and handle the case where it is hit. A low limit is a cheap safeguard against runaway output, but too low a limit silently truncates answers and tool calls.