Streaming

Without streaming, an API call returns nothing until the model has finished, which can take many seconds for a long answer. With streaming, the API sends a series of events as tokens are produced, typically over server-sent events, and your code displays or processes them as they arrive.

Streaming does not make the model generate faster overall, but it sharply reduces perceived latency: the time until the user sees something happening. For agents, streaming events can also reveal which tool the model is about to call, so the interface can show progress instead of a spinner.

The cost is more complex code. You assemble the final message from fragments, handle tool call arguments that arrive in pieces, and deal with errors that happen mid-stream. The official SDKs include helpers that do much of this assembly for you.