Chat Completions streaming events

For the complete documentation index, see llms.txt. Markdown versions of documentation pages are available by appending .md to the page URL.

Stream Chat Completions in real time. Receive chunks of completions returned from the model using server-sent events. Learn more.

chat.completion.chunk

Represents a streamed chunk of a chat completion response returned by the model, based on the provided input. Learn more.

Schema

Schema name: CreateChatCompletionStreamResponse

• id: string

  A unique identifier for the chat completion. Each chunk has the same ID.

• choices: array of object { delta, finish_reason, index, logprobs }

  A list of chat completion choices. Can contain more than one elements if n is greater than 1. Can also be empty for the last chunk if you set stream_options: {"include_usage": true}.

  • delta: object { content, function_call, refusal, 2 more }

    A chat completion delta generated by streamed model responses.

    • content: optional string or null

      The contents of the chunk message.

    • function_call: optional object { arguments, name }

      Deprecated and replaced by tool_calls. The name and arguments of a function that should be called, as generated by the model.

      • arguments: optional string

        The arguments to call the function with, as generated by the model in JSON format. Note that the model does not always generate valid JSON, and may hallucinate parameters not defined by your function schema. Validate the arguments in your code before calling your function.

      • name: optional string

        The name of the function to call.

    • refusal: optional string or null

      The refusal message generated by the model.

    • role: optional "developer" or "system" or "user" or 2 more

      The role of the author of this message.

      • "developer"

      • "system"

      • "user"

      • "assistant"

      • "tool"

    • tool_calls: optional array of object { index, id, function, type }

      • index: number

      • id: optional string

        The ID of the tool call.

      • function: optional object { arguments, name }

        • arguments: optional string

          The arguments to call the function with, as generated by the model in JSON format. Note that the model does not always generate valid JSON, and may hallucinate parameters not defined by your function schema. Validate the arguments in your code before calling your function.

        • name: optional string

          The name of the function to call.

      • type: optional "function"

        The type of the tool. Currently, only function is supported.

        • "function"

  • finish_reason: "stop" or "length" or "tool_calls" or 2 more or null

    The reason the model stopped generating tokens. This will be stop if the model hit a natural stop point or a provided stop sequence, length if the maximum number of tokens specified in the request was reached, content_filter if content was omitted due to a flag from our content filters, tool_calls if the model called a tool, or function_call (deprecated) if the model called a function.

    • "stop"

    • "length"

    • "tool_calls"

    • "content_filter"

    • "function_call"

  • index: number

    The index of the choice in the list of choices.

  • logprobs: optional object { content, refusal } or null

    Log probability information for the choice.

    • content: array of ChatCompletionTokenLogprob or null

      A list of message content tokens with log probability information.

      • token: string

        The token.

      • bytes: array of number or null

        A list of integers representing the UTF-8 bytes representation of the token. Useful in instances where characters are represented by multiple tokens and their byte representations must be combined to generate the correct text representation. Can be null if there is no bytes representation for the token.

      • logprob: number

        The log probability of this token, if it is within the top 20 most likely tokens. Otherwise, the value -9999.0 is used to signify that the token is very unlikely.

      • top_logprobs: array of object { token, bytes, logprob }

        List of the most likely tokens and their log probability, at this token position. The number of entries may be fewer than the requested top_logprobs.

        • token: string

          The token.

        • bytes: array of number or null

          A list of integers representing the UTF-8 bytes representation of the token. Useful in instances where characters are represented by multiple tokens and their byte representations must be combined to generate the correct text representation. Can be null if there is no bytes representation for the token.

        • logprob: number

          The log probability of this token, if it is within the top 20 most likely tokens. Otherwise, the value -9999.0 is used to signify that the token is very unlikely.

    • refusal: array of ChatCompletionTokenLogprob or null

      A list of message refusal tokens with log probability information.

      • token: string

        The token.

      • bytes: array of number or null

        A list of integers representing the UTF-8 bytes representation of the token. Useful in instances where characters are represented by multiple tokens and their byte representations must be combined to generate the correct text representation. Can be null if there is no bytes representation for the token.

      • logprob: number

        The log probability of this token, if it is within the top 20 most likely tokens. Otherwise, the value -9999.0 is used to signify that the token is very unlikely.

      • top_logprobs: array of object { token, bytes, logprob }

        List of the most likely tokens and their log probability, at this token position. The number of entries may be fewer than the requested top_logprobs.

• created: number

  The Unix timestamp (in seconds) of when the chat completion was created. Each chunk has the same timestamp.

• model: string

  The model to generate the completion.

• object: "chat.completion.chunk"

  The object type, which is always chat.completion.chunk.

  • "chat.completion.chunk"

• moderation: optional object { input, output } or null

  Moderation results for the request input and generated output. Present on the moderation chunk when moderated completions are requested.

  • input: object { model, results, type } or object { code, message, type }

    Moderation for the request input.

    • ModerationResults object { model, results, type }

      Successful moderation results for the request input or generated output.

      • model: string

        The moderation model used to generate the results.

      • results: array of object { categories, category_applied_input_types, category_scores, 3 more }

        A list of moderation results.

        • categories: map[boolean]

          A dictionary of moderation categories to booleans, True if the input is flagged under this category.

        • category_applied_input_types: map[array of "text" or "image"]

          Which modalities of input are reflected by the score for each category.

          • "text"

          • "image"

        • category_scores: map[number]

          A dictionary of moderation categories to scores.

        • flagged: boolean

          A boolean indicating whether the content was flagged by any category.

        • model: string

          The moderation model that produced this result.

        • type: "moderation_result"

          The object type, which was always moderation_result for successful moderation results.

          • "moderation_result"

      • type: "moderation_results"

        The object type, which is always moderation_results.

        • "moderation_results"

    • Error object { code, message, type }

      An error produced while attempting moderation.

      • code: string

        The error code.

      • message: string

        The error message.

      • type: "error"

        The object type, which is always error.

        • "error"

  • output: object { model, results, type } or object { code, message, type }

    Moderation for the generated output.

    • ModerationResults object { model, results, type }

      Successful moderation results for the request input or generated output.

      • model: string

        The moderation model used to generate the results.

      • results: array of object { categories, category_applied_input_types, category_scores, 3 more }

        A list of moderation results.

        • categories: map[boolean]

          A dictionary of moderation categories to booleans, True if the input is flagged under this category.

        • category_applied_input_types: map[array of "text" or "image"]

          Which modalities of input are reflected by the score for each category.

          • "text"

          • "image"

        • category_scores: map[number]

          A dictionary of moderation categories to scores.

        • flagged: boolean

          A boolean indicating whether the content was flagged by any category.

        • model: string

          The moderation model that produced this result.

        • type: "moderation_result"

          The object type, which was always moderation_result for successful moderation results.

          • "moderation_result"

      • type: "moderation_results"

        The object type, which is always moderation_results.

        • "moderation_results"

    • Error object { code, message, type }

      An error produced while attempting moderation.

      • code: string

        The error code.

      • message: string

        The error message.

      • type: "error"

        The object type, which is always error.

        • "error"

• obfuscation: optional string

  An obfuscation string added to normalize the size of streamed chunks as a mitigation to certain side-channel attacks. The field is included by default and omitted when stream_options.include_obfuscation is false.

• service_tier: optional "auto" or "default" or "flex" or 3 more or null

  Specifies the processing type used for serving the request.

  • If set to 'auto', then the request will be processed with the service tier configured in the Project settings. Unless otherwise configured, the Project will use 'default'.
  • If set to 'default', then the request will be processed with the standard pricing and performance for the selected model.
  • If set to 'flex', then the request will be processed with the Flex Processing service tier.
  • To opt-in to Fast mode at the request level, include the service_tier=fast or service_tier=priority parameter for Responses or Chat Completions. The response will show service_tier=priority regardless of if you specify service_tier=fast or priority in your request.
  • When not set, the default behavior is 'auto'.

  When the service_tier parameter is set, the response body will include the service_tier value based on the processing mode actually used to serve the request. This response value may be different from the value set in the parameter.

  • "auto"

  • "default"

  • "flex"

  • "scale"

  • "priority"

  • "fast"

• system_fingerprint: optional string

  This fingerprint represents the backend configuration that the model runs with. Can be used in conjunction with the seed request parameter to understand when backend changes have been made that might impact determinism.

• usage: optional CompletionUsage or null

  An optional field that will only be present when you set stream_options: {"include_usage": true} in your request. When present, it contains a null value except for the last chunk which contains the token usage statistics for the entire request.

  NOTE: If the stream is interrupted or cancelled, you may not receive the final usage chunk which contains the total token usage for the request.

  • completion_tokens: number

    Number of tokens in the generated completion.

  • prompt_tokens: number

    Number of tokens in the prompt.

  • total_tokens: number

    Total number of tokens used in the request (prompt + completion).

  • completion_tokens_details: optional object { accepted_prediction_tokens, audio_tokens, reasoning_tokens, 2 more }

    Breakdown of tokens used in a completion.

    • accepted_prediction_tokens: optional number

      When using Predicted Outputs, the number of tokens in the prediction that appeared in the completion.

    • audio_tokens: optional number

      Audio input tokens generated by the model.

    • reasoning_tokens: optional number

      Tokens generated by the model for reasoning.

    • rejected_prediction_tokens: optional number

      When using Predicted Outputs, the number of tokens in the prediction that did not appear in the completion. However, like reasoning tokens, these tokens are still counted in the total completion tokens for purposes of billing, output, and context window limits.

    • text_tokens: optional number

      Text output tokens generated by the model.

  • prompt_tokens_details: optional object { audio_tokens, cache_write_tokens, cached_tokens, 2 more }

    Breakdown of tokens used in the prompt.

    • audio_tokens: optional number

      Audio input tokens present in the prompt.

    • cache_write_tokens: optional number

      The unadjusted number of prompt tokens written to cache.

    • cached_tokens: optional number

      Cached tokens present in the prompt.

    • image_tokens: optional number

      Image input tokens present in the prompt.

    • text_tokens: optional number

      Text input tokens present in the prompt.

Example

{"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"gpt-6-astra", "system_fingerprint": "fp_44709d6fcb", "choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}],"obfuscation":"r4N7vQ2m"}

{"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"gpt-6-astra", "system_fingerprint": "fp_44709d6fcb", "choices":[{"index":0,"delta":{"content":"Hello"},"logprobs":null,"finish_reason":null}],"obfuscation":"p9K3xT6w"}

....

{"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"gpt-6-astra", "system_fingerprint": "fp_44709d6fcb", "choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"stop"}],"obfuscation":""}

Previously: resources/chat/subresources/completions/streaming-events.md