NovitàCorsia Eco ora su 91 modelli: lo stesso modello a un prezzo più basso

Valuta

Ricarichi sempre in euro. Le altre valute sono indicate al tasso BCE del giorno. Tassi di riferimento BCE

AccediInizia

Documentazionev1

Menu della documentazioneStreaming

Streaming

Stream tokens as server-sent events on every chat endpoint, get usage at the end of a stream, and handle errors mid-stream.

La documentazione è in inglese.

Set "stream": true and the answer arrives as server-sent events (text/event-stream) while the model writes it. The gateway passes the events on as the model provider sends them, in the format of the endpoint you called, with model set to the model's public ID. Streaming works on /v1/chat/completions, /v1/messages and /v1/responses.

Chat completions

Each event is a data: line with a chat.completion.chunk. The text is in choices[0].delta.content. The stream ends with data: [DONE].

Stream
data: {"id":"chatcmpl-8f1c","object":"chat.completion.chunk","created":1795075200,"model":"openai/gpt-6-astra","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]} data: {"id":"chatcmpl-8f1c","object":"chat.completion.chunk","created":1795075200,"model":"openai/gpt-6-astra","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]} data: {"id":"chatcmpl-8f1c","object":"chat.completion.chunk","created":1795075200,"model":"openai/gpt-6-astra","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]} data: {"id":"chatcmpl-8f1c","object":"chat.completion.chunk","created":1795075200,"model":"openai/gpt-6-astra","choices":[],"usage":{"prompt_tokens":9,"completion_tokens":10,"total_tokens":19}} data: [DONE]

Usage at the end of a stream

To get the token counts, set stream_options.include_usage to true. The last chunk before [DONE] then has an empty choices array and a usage object, as above. Without it, the gateway leaves usage out of the stream. You are billed the same either way.

stream = client.chat.completions.create(    model="openai/gpt-6-astra",    messages=[{"role": "user", "content": "Hello"}],    stream=True,    stream_options={"include_usage": True},)for chunk in stream:    if chunk.choices and chunk.choices[0].delta.content:        print(chunk.choices[0].delta.content, end="")    if chunk.usage:        print("\ntokens:", chunk.usage.total_tokens)

Messages (Anthropic format)

On /v1/messages, every event has a name: message_start (with the input tokens), content_block_start, content_block_delta (the text), content_block_stop, message_delta (the output tokens and the stop reason) and message_stop. Usage is always included.

Stream
event: message_startdata: {"type":"message_start","message":{"id":"msg_01Xf","type":"message","role":"assistant","model":"anthropic/claude-sonnet-5.5","content":[],"stop_reason":null,"usage":{"input_tokens":9,"output_tokens":1}}} event: content_block_deltadata: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Hello"}} event: message_deltadata: {"type":"message_delta","delta":{"stop_reason":"end_turn","stop_sequence":null},"usage":{"output_tokens":12}} event: message_stopdata: {"type":"message_stop"}

Responses

On /v1/responses, events are named after their type, such as response.created, response.output_text.delta and response.completed. The response.completed event carries the usage in response.usage.

Keep-alive lines

While a model is thinking, comment lines (: keep-alive) can arrive between events. SSE clients and the SDKs skip them.

Errors during a stream

Errors before the stream starts come back as a normal JSON error with an HTTP status, such as 402 or 429. Once the stream has started, the status is already 200, so an error arrives as an event:

  • Chat completions: a data: line with an error object. The code is stream_interrupted when the connection to the model was lost.
  • /v1/messages: an event: error with an Anthropic error object.
  • /v1/responses: an event: error, or a response.failed event.
Stream
data: {"error":{"message":"The model stopped responding. Try again.","type":"server_error","param":null,"code":"stream_interrupted"}}

The gateway ends a stream when no data arrives from the model for two minutes. There is no fallback to another route once data has reached you, so retry the request yourself.

How a stream is billed

We bill on the usage the model reports at the end of the stream. If you close the connection early, the gateway stops the request and bills the input and the output written so far. If a stream breaks off without usage, we estimate the tokens from the text, at about four characters per token. A stream that fails before any output costs nothing, except on /v1/messages: there the model reports the input tokens at the very start, in message_start, and from that moment the input is billed. See Pricing & usage.