Realtime

Domain Types

Audio Transcription

class AudioTranscription
- delay: :minimal | :low | :medium | 2 more
  
  Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
  - :minimal
  - :low
  - :medium
  - :high
  - :xhigh
- language: String
  
  The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
- model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
  
  The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
  - String = String
  - Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
    
    The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
    - :"whisper-1"
    - :"gpt-4o-mini-transcribe"
    - :"gpt-4o-mini-transcribe-2025-12-15"
    - :"gpt-4o-transcribe"
    - :"gpt-4o-transcribe-diarize"
    - :"gpt-realtime-whisper"
- prompt: String
  
  An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.

Conversation Created Event

class ConversationCreatedEvent

Returned when a conversation is created. Emitted right after session creation.
- conversation: Conversation{ id, object}
  
  The conversation resource.
  - id: String
    
    The unique ID of the conversation.
  - object: :"realtime.conversation"
    
    The object type, must be realtime.conversation.
    - :"realtime.conversation"
- event_id: String
  
  The unique ID of the server event.
- type: :"conversation.created"
  
  The event type, must be conversation.created.
  - :"conversation.created"

Conversation Item

ConversationItem = RealtimeConversationItemSystemMessage | RealtimeConversationItemUserMessage | RealtimeConversationItemAssistantMessage | 6 more

A single item within a Realtime conversation.
- class RealtimeConversationItemSystemMessage
  
  A system message in a Realtime conversation can be used to provide additional context or instructions to the model. This is similar but distinct from the instruction prompt provided at the start of a conversation, as system messages can be added at any point in the conversation. For major changes to the conversation's behavior, use instructions, but for smaller updates (e.g. "the user is now asking about a different topic"), use system messages.
  - content: Array[Content{ text, type}]
    
    The content of the message.
    - text: String
      
      The text content.
    - type: :input_text
      
      The content type. Always input_text for system messages.
      - :input_text
  - role: :system
    
    The role of the message sender. Always system.
    - :system
  - type: :message
    
    The type of the item. Always message.
    - :message
  - id: String
    
    The unique ID of the item. This may be provided by the client or generated by the server.
  - object: :"realtime.item"
    
    Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
    - :"realtime.item"
  - status: :completed | :incomplete | :in_progress
    
    The status of the item. Has no effect on the conversation.
    - :completed
    - :incomplete
    - :in_progress
- class RealtimeConversationItemUserMessage
  
  A user message item in a Realtime conversation.
  - content: Array[Content{ audio, detail, image_url, 3 more}]
    
    The content of the message.
    - audio: String
      
      Base64-encoded audio bytes (for input_audio), these will be parsed as the format specified in the session input audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
    - detail: :auto | :low | :high
      
      The detail level of the image (for input_image). auto will default to high.
      - :auto
      - :low
      - :high
    - image_url: String
      
      Base64-encoded image bytes (for input_image) as a data URI. For example data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA.... Supported formats are PNG and JPEG.
    - text: String
      
      The text content (for input_text).
    - transcript: String
      
      Transcript of the audio (for input_audio). This is not sent to the model, but will be attached to the message item for reference.
    - type: :input_text | :input_audio | :input_image
      
      The content type (input_text, input_audio, or input_image).
      - :input_text
      - :input_audio
      - :input_image
  - role: :user
    
    The role of the message sender. Always user.
    - :user
  - type: :message
    
    The type of the item. Always message.
    - :message
  - id: String
    
    The unique ID of the item. This may be provided by the client or generated by the server.
  - object: :"realtime.item"
    
    Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
    - :"realtime.item"
  - status: :completed | :incomplete | :in_progress
    
    The status of the item. Has no effect on the conversation.
    - :completed
    - :incomplete
    - :in_progress
- class RealtimeConversationItemAssistantMessage
  
  An assistant message item in a Realtime conversation.
  - content: Array[Content{ audio, text, transcript, type}]
    
    The content of the message.
    - audio: String
      
      Base64-encoded audio bytes, these will be parsed as the format specified in the session output audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
    - text: String
      
      The text content.
    - transcript: String
      
      The transcript of the audio content, this will always be present if the output type is audio.
    - type: :output_text | :output_audio
      
      The content type, output_text or output_audio depending on the session output_modalities configuration.
      - :output_text
      - :output_audio
  - role: :assistant
    
    The role of the message sender. Always assistant.
    - :assistant
  - type: :message
    
    The type of the item. Always message.
    - :message
  - id: String
    
    The unique ID of the item. This may be provided by the client or generated by the server.
  - object: :"realtime.item"
    
    Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
    - :"realtime.item"
  - status: :completed | :incomplete | :in_progress
    
    The status of the item. Has no effect on the conversation.
    - :completed
    - :incomplete
    - :in_progress
- class RealtimeConversationItemFunctionCall
  
  A function call item in a Realtime conversation.
  - arguments: String
    
    The arguments of the function call. This is a JSON-encoded string representing the arguments passed to the function, for example {"arg1": "value1", "arg2": 42}.
  - name: String
    
    The name of the function being called.
  - type: :function_call
    
    The type of the item. Always function_call.
    - :function_call
  - id: String
    
    The unique ID of the item. This may be provided by the client or generated by the server.
  - call_id: String
    
    The ID of the function call.
  - object: :"realtime.item"
    
    Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
    - :"realtime.item"
  - status: :completed | :incomplete | :in_progress
    
    The status of the item. Has no effect on the conversation.
    - :completed
    - :incomplete
    - :in_progress
- class RealtimeConversationItemFunctionCallOutput
  
  A function call output item in a Realtime conversation.
  - call_id: String
    
    The ID of the function call this output is for.
  - output: String
    
    The output of the function call, this is free text and can contain any information or simply be empty.
  - type: :function_call_output
    
    The type of the item. Always function_call_output.
    - :function_call_output
  - id: String
    
    The unique ID of the item. This may be provided by the client or generated by the server.
  - object: :"realtime.item"
    
    Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
    - :"realtime.item"
  - status: :completed | :incomplete | :in_progress
    
    The status of the item. Has no effect on the conversation.
    - :completed
    - :incomplete
    - :in_progress
- class RealtimeMcpApprovalResponse
  
  A Realtime item responding to an MCP approval request.
  - id: String
    
    The unique ID of the approval response.
  - approval_request_id: String
    
    The ID of the approval request being answered.
  - approve: bool
    
    Whether the request was approved.
  - type: :mcp_approval_response
    
    The type of the item. Always mcp_approval_response.
    - :mcp_approval_response
  - reason: String
    
    Optional reason for the decision.
- class RealtimeMcpListTools
  
  A Realtime item listing tools available on an MCP server.
  - server_label: String
    
    The label of the MCP server.
  - tools: Array[Tool{ input_schema, name, annotations, description}]
    
    The tools available on the server.
    - input_schema: untyped
      
      The JSON schema describing the tool's input.
    - name: String
      
      The name of the tool.
    - annotations: untyped
      
      Additional annotations about the tool.
    - description: String
      
      The description of the tool.
  - type: :mcp_list_tools
    
    The type of the item. Always mcp_list_tools.
    - :mcp_list_tools
  - id: String
    
    The unique ID of the list.
- class RealtimeMcpToolCall
  
  A Realtime item representing an invocation of a tool on an MCP server.
  - id: String
    
    The unique ID of the tool call.
  - arguments: String
    
    A JSON string of the arguments passed to the tool.
  - name: String
    
    The name of the tool that was run.
  - server_label: String
    
    The label of the MCP server running the tool.
  - type: :mcp_call
    
    The type of the item. Always mcp_call.
    - :mcp_call
  - approval_request_id: String
    
    The ID of an associated approval request, if any.
  - error: RealtimeMcpProtocolError | RealtimeMcpToolExecutionError | RealtimeMcphttpError
    
    The error from the tool call, if any.
    - class RealtimeMcpProtocolError
      - code: Integer
      - message: String
      - type: :protocol_error
        
        :protocol_error
    - class RealtimeMcpToolExecutionError
      - message: String
      - type: :tool_execution_error
        
        :tool_execution_error
    - class RealtimeMcphttpError
      - code: Integer
      - message: String
      - type: :http_error
        
        :http_error
  - output: String
    
    The output from the tool call.
- class RealtimeMcpApprovalRequest
  
  A Realtime item requesting human approval of a tool invocation.
  - id: String
    
    The unique ID of the approval request.
  - arguments: String
    
    A JSON string of arguments for the tool.
  - name: String
    
    The name of the tool to run.
  - server_label: String
    
    The label of the MCP server making the request.
  - type: :mcp_approval_request
    
    The type of the item. Always mcp_approval_request.
    - :mcp_approval_request

Conversation Item Added

class ConversationItemAdded

Sent by the server when an Item is added to the default Conversation. This can happen in several cases:
- When the client sends a conversation.item.create event.
- When the input audio buffer is committed. In this case the item will be a user message containing the audio from the buffer.
- When the model is generating a Response. In this case the conversation.item.added event will be sent when the model starts generating a specific Item, and thus it will not yet have any content (and status will be in_progress).
The event will include the full content of the Item (except when model is generating a Response) except for audio data, which can be retrieved separately with a conversation.item.retrieve event if necessary.
- event_id: String
  
  The unique ID of the server event.
- item: ConversationItem
  
  A single item within a Realtime conversation.
  - class RealtimeConversationItemSystemMessage
    
    A system message in a Realtime conversation can be used to provide additional context or instructions to the model. This is similar but distinct from the instruction prompt provided at the start of a conversation, as system messages can be added at any point in the conversation. For major changes to the conversation's behavior, use instructions, but for smaller updates (e.g. "the user is now asking about a different topic"), use system messages.
    - content: Array[Content{ text, type}]
      
      The content of the message.
      - text: String
        
        The text content.
      - type: :input_text
        
        The content type. Always input_text for system messages.
        
        :input_text
    - role: :system
      
      The role of the message sender. Always system.
      - :system
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemUserMessage
    
    A user message item in a Realtime conversation.
    - content: Array[Content{ audio, detail, image_url, 3 more}]
      
      The content of the message.
      - audio: String
        
        Base64-encoded audio bytes (for input_audio), these will be parsed as the format specified in the session input audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
      - detail: :auto | :low | :high
        
        The detail level of the image (for input_image). auto will default to high.
        
        :auto
        
        :low
        
        :high
      - image_url: String
        
        Base64-encoded image bytes (for input_image) as a data URI. For example data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA.... Supported formats are PNG and JPEG.
      - text: String
        
        The text content (for input_text).
      - transcript: String
        
        Transcript of the audio (for input_audio). This is not sent to the model, but will be attached to the message item for reference.
      - type: :input_text | :input_audio | :input_image
        
        The content type (input_text, input_audio, or input_image).
        
        :input_text
        
        :input_audio
        
        :input_image
    - role: :user
      
      The role of the message sender. Always user.
      - :user
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemAssistantMessage
    
    An assistant message item in a Realtime conversation.
    - content: Array[Content{ audio, text, transcript, type}]
      
      The content of the message.
      - audio: String
        
        Base64-encoded audio bytes, these will be parsed as the format specified in the session output audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
      - text: String
        
        The text content.
      - transcript: String
        
        The transcript of the audio content, this will always be present if the output type is audio.
      - type: :output_text | :output_audio
        
        The content type, output_text or output_audio depending on the session output_modalities configuration.
        
        :output_text
        
        :output_audio
    - role: :assistant
      
      The role of the message sender. Always assistant.
      - :assistant
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemFunctionCall
    
    A function call item in a Realtime conversation.
    - arguments: String
      
      The arguments of the function call. This is a JSON-encoded string representing the arguments passed to the function, for example {"arg1": "value1", "arg2": 42}.
    - name: String
      
      The name of the function being called.
    - type: :function_call
      
      The type of the item. Always function_call.
      - :function_call
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - call_id: String
      
      The ID of the function call.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemFunctionCallOutput
    
    A function call output item in a Realtime conversation.
    - call_id: String
      
      The ID of the function call this output is for.
    - output: String
      
      The output of the function call, this is free text and can contain any information or simply be empty.
    - type: :function_call_output
      
      The type of the item. Always function_call_output.
      - :function_call_output
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeMcpApprovalResponse
    
    A Realtime item responding to an MCP approval request.
    - id: String
      
      The unique ID of the approval response.
    - approval_request_id: String
      
      The ID of the approval request being answered.
    - approve: bool
      
      Whether the request was approved.
    - type: :mcp_approval_response
      
      The type of the item. Always mcp_approval_response.
      - :mcp_approval_response
    - reason: String
      
      Optional reason for the decision.
  - class RealtimeMcpListTools
    
    A Realtime item listing tools available on an MCP server.
    - server_label: String
      
      The label of the MCP server.
    - tools: Array[Tool{ input_schema, name, annotations, description}]
      
      The tools available on the server.
      - input_schema: untyped
        
        The JSON schema describing the tool's input.
      - name: String
        
        The name of the tool.
      - annotations: untyped
        
        Additional annotations about the tool.
      - description: String
        
        The description of the tool.
    - type: :mcp_list_tools
      
      The type of the item. Always mcp_list_tools.
      - :mcp_list_tools
    - id: String
      
      The unique ID of the list.
  - class RealtimeMcpToolCall
    
    A Realtime item representing an invocation of a tool on an MCP server.
    - id: String
      
      The unique ID of the tool call.
    - arguments: String
      
      A JSON string of the arguments passed to the tool.
    - name: String
      
      The name of the tool that was run.
    - server_label: String
      
      The label of the MCP server running the tool.
    - type: :mcp_call
      
      The type of the item. Always mcp_call.
      - :mcp_call
    - approval_request_id: String
      
      The ID of an associated approval request, if any.
    - error: RealtimeMcpProtocolError | RealtimeMcpToolExecutionError | RealtimeMcphttpError
      
      The error from the tool call, if any.
      - class RealtimeMcpProtocolError
        
        code: Integer
        
        message: String
        
        type: :protocol_error
        
        :protocol_error
      - class RealtimeMcpToolExecutionError
        
        message: String
        
        type: :tool_execution_error
        
        :tool_execution_error
      - class RealtimeMcphttpError
        
        code: Integer
        
        message: String
        
        type: :http_error
        
        :http_error
    - output: String
      
      The output from the tool call.
  - class RealtimeMcpApprovalRequest
    
    A Realtime item requesting human approval of a tool invocation.
    - id: String
      
      The unique ID of the approval request.
    - arguments: String
      
      A JSON string of arguments for the tool.
    - name: String
      
      The name of the tool to run.
    - server_label: String
      
      The label of the MCP server making the request.
    - type: :mcp_approval_request
      
      The type of the item. Always mcp_approval_request.
      - :mcp_approval_request
- type: :"conversation.item.added"
  
  The event type, must be conversation.item.added.
  - :"conversation.item.added"
- previous_item_id: String
  
  The ID of the item that precedes this one, if any. This is used to maintain ordering when items are inserted.

Conversation Item Create Event

class ConversationItemCreateEvent

Add a new Item to the Conversation's context, including messages, function calls, and function call responses. This event can be used both to populate a "history" of the conversation and to add new items mid-stream, but has the current limitation that it cannot populate assistant audio messages.

If successful, the server will respond with a conversation.item.created event, otherwise an error event will be sent.
- item: ConversationItem
  
  A single item within a Realtime conversation.
  - class RealtimeConversationItemSystemMessage
    
    A system message in a Realtime conversation can be used to provide additional context or instructions to the model. This is similar but distinct from the instruction prompt provided at the start of a conversation, as system messages can be added at any point in the conversation. For major changes to the conversation's behavior, use instructions, but for smaller updates (e.g. "the user is now asking about a different topic"), use system messages.
    - content: Array[Content{ text, type}]
      
      The content of the message.
      - text: String
        
        The text content.
      - type: :input_text
        
        The content type. Always input_text for system messages.
        
        :input_text
    - role: :system
      
      The role of the message sender. Always system.
      - :system
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemUserMessage
    
    A user message item in a Realtime conversation.
    - content: Array[Content{ audio, detail, image_url, 3 more}]
      
      The content of the message.
      - audio: String
        
        Base64-encoded audio bytes (for input_audio), these will be parsed as the format specified in the session input audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
      - detail: :auto | :low | :high
        
        The detail level of the image (for input_image). auto will default to high.
        
        :auto
        
        :low
        
        :high
      - image_url: String
        
        Base64-encoded image bytes (for input_image) as a data URI. For example data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA.... Supported formats are PNG and JPEG.
      - text: String
        
        The text content (for input_text).
      - transcript: String
        
        Transcript of the audio (for input_audio). This is not sent to the model, but will be attached to the message item for reference.
      - type: :input_text | :input_audio | :input_image
        
        The content type (input_text, input_audio, or input_image).
        
        :input_text
        
        :input_audio
        
        :input_image
    - role: :user
      
      The role of the message sender. Always user.
      - :user
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemAssistantMessage
    
    An assistant message item in a Realtime conversation.
    - content: Array[Content{ audio, text, transcript, type}]
      
      The content of the message.
      - audio: String
        
        Base64-encoded audio bytes, these will be parsed as the format specified in the session output audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
      - text: String
        
        The text content.
      - transcript: String
        
        The transcript of the audio content, this will always be present if the output type is audio.
      - type: :output_text | :output_audio
        
        The content type, output_text or output_audio depending on the session output_modalities configuration.
        
        :output_text
        
        :output_audio
    - role: :assistant
      
      The role of the message sender. Always assistant.
      - :assistant
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemFunctionCall
    
    A function call item in a Realtime conversation.
    - arguments: String
      
      The arguments of the function call. This is a JSON-encoded string representing the arguments passed to the function, for example {"arg1": "value1", "arg2": 42}.
    - name: String
      
      The name of the function being called.
    - type: :function_call
      
      The type of the item. Always function_call.
      - :function_call
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - call_id: String
      
      The ID of the function call.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemFunctionCallOutput
    
    A function call output item in a Realtime conversation.
    - call_id: String
      
      The ID of the function call this output is for.
    - output: String
      
      The output of the function call, this is free text and can contain any information or simply be empty.
    - type: :function_call_output
      
      The type of the item. Always function_call_output.
      - :function_call_output
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeMcpApprovalResponse
    
    A Realtime item responding to an MCP approval request.
    - id: String
      
      The unique ID of the approval response.
    - approval_request_id: String
      
      The ID of the approval request being answered.
    - approve: bool
      
      Whether the request was approved.
    - type: :mcp_approval_response
      
      The type of the item. Always mcp_approval_response.
      - :mcp_approval_response
    - reason: String
      
      Optional reason for the decision.
  - class RealtimeMcpListTools
    
    A Realtime item listing tools available on an MCP server.
    - server_label: String
      
      The label of the MCP server.
    - tools: Array[Tool{ input_schema, name, annotations, description}]
      
      The tools available on the server.
      - input_schema: untyped
        
        The JSON schema describing the tool's input.
      - name: String
        
        The name of the tool.
      - annotations: untyped
        
        Additional annotations about the tool.
      - description: String
        
        The description of the tool.
    - type: :mcp_list_tools
      
      The type of the item. Always mcp_list_tools.
      - :mcp_list_tools
    - id: String
      
      The unique ID of the list.
  - class RealtimeMcpToolCall
    
    A Realtime item representing an invocation of a tool on an MCP server.
    - id: String
      
      The unique ID of the tool call.
    - arguments: String
      
      A JSON string of the arguments passed to the tool.
    - name: String
      
      The name of the tool that was run.
    - server_label: String
      
      The label of the MCP server running the tool.
    - type: :mcp_call
      
      The type of the item. Always mcp_call.
      - :mcp_call
    - approval_request_id: String
      
      The ID of an associated approval request, if any.
    - error: RealtimeMcpProtocolError | RealtimeMcpToolExecutionError | RealtimeMcphttpError
      
      The error from the tool call, if any.
      - class RealtimeMcpProtocolError
        
        code: Integer
        
        message: String
        
        type: :protocol_error
        
        :protocol_error
      - class RealtimeMcpToolExecutionError
        
        message: String
        
        type: :tool_execution_error
        
        :tool_execution_error
      - class RealtimeMcphttpError
        
        code: Integer
        
        message: String
        
        type: :http_error
        
        :http_error
    - output: String
      
      The output from the tool call.
  - class RealtimeMcpApprovalRequest
    
    A Realtime item requesting human approval of a tool invocation.
    - id: String
      
      The unique ID of the approval request.
    - arguments: String
      
      A JSON string of arguments for the tool.
    - name: String
      
      The name of the tool to run.
    - server_label: String
      
      The label of the MCP server making the request.
    - type: :mcp_approval_request
      
      The type of the item. Always mcp_approval_request.
      - :mcp_approval_request
- type: :"conversation.item.create"
  
  The event type, must be conversation.item.create.
  - :"conversation.item.create"
- event_id: String
  
  Optional client-generated ID used to identify this event.
- previous_item_id: String
  
  The ID of the preceding item after which the new item will be inserted. If not set, the new item will be appended to the end of the conversation.
  
  If set to root, the new item will be added to the beginning of the conversation.
  
  If set to an existing ID, it allows an item to be inserted mid-conversation. If the ID cannot be found, an error will be returned and the item will not be added.

Conversation Item Created Event

class ConversationItemCreatedEvent

Returned when a conversation item is created. There are several scenarios that produce this event:
- The server is generating a Response, which if successful will produce either one or two Items, which will be of type message (role assistant) or type function_call.
- The input audio buffer has been committed, either by the client or the server (in server_vad mode). The server will take the content of the input audio buffer and add it to a new user message Item.
- The client has sent a conversation.item.create event to add a new Item to the Conversation.
- event_id: String
  
  The unique ID of the server event.
- item: ConversationItem
  
  A single item within a Realtime conversation.
  - class RealtimeConversationItemSystemMessage
    
    A system message in a Realtime conversation can be used to provide additional context or instructions to the model. This is similar but distinct from the instruction prompt provided at the start of a conversation, as system messages can be added at any point in the conversation. For major changes to the conversation's behavior, use instructions, but for smaller updates (e.g. "the user is now asking about a different topic"), use system messages.
    - content: Array[Content{ text, type}]
      
      The content of the message.
      - text: String
        
        The text content.
      - type: :input_text
        
        The content type. Always input_text for system messages.
        
        :input_text
    - role: :system
      
      The role of the message sender. Always system.
      - :system
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemUserMessage
    
    A user message item in a Realtime conversation.
    - content: Array[Content{ audio, detail, image_url, 3 more}]
      
      The content of the message.
      - audio: String
        
        Base64-encoded audio bytes (for input_audio), these will be parsed as the format specified in the session input audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
      - detail: :auto | :low | :high
        
        The detail level of the image (for input_image). auto will default to high.
        
        :auto
        
        :low
        
        :high
      - image_url: String
        
        Base64-encoded image bytes (for input_image) as a data URI. For example data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA.... Supported formats are PNG and JPEG.
      - text: String
        
        The text content (for input_text).
      - transcript: String
        
        Transcript of the audio (for input_audio). This is not sent to the model, but will be attached to the message item for reference.
      - type: :input_text | :input_audio | :input_image
        
        The content type (input_text, input_audio, or input_image).
        
        :input_text
        
        :input_audio
        
        :input_image
    - role: :user
      
      The role of the message sender. Always user.
      - :user
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemAssistantMessage
    
    An assistant message item in a Realtime conversation.
    - content: Array[Content{ audio, text, transcript, type}]
      
      The content of the message.
      - audio: String
        
        Base64-encoded audio bytes, these will be parsed as the format specified in the session output audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
      - text: String
        
        The text content.
      - transcript: String
        
        The transcript of the audio content, this will always be present if the output type is audio.
      - type: :output_text | :output_audio
        
        The content type, output_text or output_audio depending on the session output_modalities configuration.
        
        :output_text
        
        :output_audio
    - role: :assistant
      
      The role of the message sender. Always assistant.
      - :assistant
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemFunctionCall
    
    A function call item in a Realtime conversation.
    - arguments: String
      
      The arguments of the function call. This is a JSON-encoded string representing the arguments passed to the function, for example {"arg1": "value1", "arg2": 42}.
    - name: String
      
      The name of the function being called.
    - type: :function_call
      
      The type of the item. Always function_call.
      - :function_call
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - call_id: String
      
      The ID of the function call.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemFunctionCallOutput
    
    A function call output item in a Realtime conversation.
    - call_id: String
      
      The ID of the function call this output is for.
    - output: String
      
      The output of the function call, this is free text and can contain any information or simply be empty.
    - type: :function_call_output
      
      The type of the item. Always function_call_output.
      - :function_call_output
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeMcpApprovalResponse
    
    A Realtime item responding to an MCP approval request.
    - id: String
      
      The unique ID of the approval response.
    - approval_request_id: String
      
      The ID of the approval request being answered.
    - approve: bool
      
      Whether the request was approved.
    - type: :mcp_approval_response
      
      The type of the item. Always mcp_approval_response.
      - :mcp_approval_response
    - reason: String
      
      Optional reason for the decision.
  - class RealtimeMcpListTools
    
    A Realtime item listing tools available on an MCP server.
    - server_label: String
      
      The label of the MCP server.
    - tools: Array[Tool{ input_schema, name, annotations, description}]
      
      The tools available on the server.
      - input_schema: untyped
        
        The JSON schema describing the tool's input.
      - name: String
        
        The name of the tool.
      - annotations: untyped
        
        Additional annotations about the tool.
      - description: String
        
        The description of the tool.
    - type: :mcp_list_tools
      
      The type of the item. Always mcp_list_tools.
      - :mcp_list_tools
    - id: String
      
      The unique ID of the list.
  - class RealtimeMcpToolCall
    
    A Realtime item representing an invocation of a tool on an MCP server.
    - id: String
      
      The unique ID of the tool call.
    - arguments: String
      
      A JSON string of the arguments passed to the tool.
    - name: String
      
      The name of the tool that was run.
    - server_label: String
      
      The label of the MCP server running the tool.
    - type: :mcp_call
      
      The type of the item. Always mcp_call.
      - :mcp_call
    - approval_request_id: String
      
      The ID of an associated approval request, if any.
    - error: RealtimeMcpProtocolError | RealtimeMcpToolExecutionError | RealtimeMcphttpError
      
      The error from the tool call, if any.
      - class RealtimeMcpProtocolError
        
        code: Integer
        
        message: String
        
        type: :protocol_error
        
        :protocol_error
      - class RealtimeMcpToolExecutionError
        
        message: String
        
        type: :tool_execution_error
        
        :tool_execution_error
      - class RealtimeMcphttpError
        
        code: Integer
        
        message: String
        
        type: :http_error
        
        :http_error
    - output: String
      
      The output from the tool call.
  - class RealtimeMcpApprovalRequest
    
    A Realtime item requesting human approval of a tool invocation.
    - id: String
      
      The unique ID of the approval request.
    - arguments: String
      
      A JSON string of arguments for the tool.
    - name: String
      
      The name of the tool to run.
    - server_label: String
      
      The label of the MCP server making the request.
    - type: :mcp_approval_request
      
      The type of the item. Always mcp_approval_request.
      - :mcp_approval_request
- type: :"conversation.item.created"
  
  The event type, must be conversation.item.created.
  - :"conversation.item.created"
- previous_item_id: String
  
  The ID of the preceding item in the Conversation context, allows the client to understand the order of the conversation. Can be null if the item has no predecessor.

Conversation Item Delete Event

class ConversationItemDeleteEvent

Send this event when you want to remove any item from the conversation history. The server will respond with a conversation.item.deleted event, unless the item does not exist in the conversation history, in which case the server will respond with an error.
- item_id: String
  
  The ID of the item to delete.
- type: :"conversation.item.delete"
  
  The event type, must be conversation.item.delete.
  - :"conversation.item.delete"
- event_id: String
  
  Optional client-generated ID used to identify this event.

Conversation Item Deleted Event

class ConversationItemDeletedEvent

Returned when an item in the conversation is deleted by the client with a conversation.item.delete event. This event is used to synchronize the server's understanding of the conversation history with the client's view.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the item that was deleted.
- type: :"conversation.item.deleted"
  
  The event type, must be conversation.item.deleted.
  - :"conversation.item.deleted"

Conversation Item Done

class ConversationItemDone

Returned when a conversation item is finalized.

The event will include the full content of the Item except for audio data, which can be retrieved separately with a conversation.item.retrieve event if needed.
- event_id: String
  
  The unique ID of the server event.
- item: ConversationItem
  
  A single item within a Realtime conversation.
  - class RealtimeConversationItemSystemMessage
    
    A system message in a Realtime conversation can be used to provide additional context or instructions to the model. This is similar but distinct from the instruction prompt provided at the start of a conversation, as system messages can be added at any point in the conversation. For major changes to the conversation's behavior, use instructions, but for smaller updates (e.g. "the user is now asking about a different topic"), use system messages.
    - content: Array[Content{ text, type}]
      
      The content of the message.
      - text: String
        
        The text content.
      - type: :input_text
        
        The content type. Always input_text for system messages.
        
        :input_text
    - role: :system
      
      The role of the message sender. Always system.
      - :system
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemUserMessage
    
    A user message item in a Realtime conversation.
    - content: Array[Content{ audio, detail, image_url, 3 more}]
      
      The content of the message.
      - audio: String
        
        Base64-encoded audio bytes (for input_audio), these will be parsed as the format specified in the session input audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
      - detail: :auto | :low | :high
        
        The detail level of the image (for input_image). auto will default to high.
        
        :auto
        
        :low
        
        :high
      - image_url: String
        
        Base64-encoded image bytes (for input_image) as a data URI. For example data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA.... Supported formats are PNG and JPEG.
      - text: String
        
        The text content (for input_text).
      - transcript: String
        
        Transcript of the audio (for input_audio). This is not sent to the model, but will be attached to the message item for reference.
      - type: :input_text | :input_audio | :input_image
        
        The content type (input_text, input_audio, or input_image).
        
        :input_text
        
        :input_audio
        
        :input_image
    - role: :user
      
      The role of the message sender. Always user.
      - :user
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemAssistantMessage
    
    An assistant message item in a Realtime conversation.
    - content: Array[Content{ audio, text, transcript, type}]
      
      The content of the message.
      - audio: String
        
        Base64-encoded audio bytes, these will be parsed as the format specified in the session output audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
      - text: String
        
        The text content.
      - transcript: String
        
        The transcript of the audio content, this will always be present if the output type is audio.
      - type: :output_text | :output_audio
        
        The content type, output_text or output_audio depending on the session output_modalities configuration.
        
        :output_text
        
        :output_audio
    - role: :assistant
      
      The role of the message sender. Always assistant.
      - :assistant
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemFunctionCall
    
    A function call item in a Realtime conversation.
    - arguments: String
      
      The arguments of the function call. This is a JSON-encoded string representing the arguments passed to the function, for example {"arg1": "value1", "arg2": 42}.
    - name: String
      
      The name of the function being called.
    - type: :function_call
      
      The type of the item. Always function_call.
      - :function_call
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - call_id: String
      
      The ID of the function call.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemFunctionCallOutput
    
    A function call output item in a Realtime conversation.
    - call_id: String
      
      The ID of the function call this output is for.
    - output: String
      
      The output of the function call, this is free text and can contain any information or simply be empty.
    - type: :function_call_output
      
      The type of the item. Always function_call_output.
      - :function_call_output
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeMcpApprovalResponse
    
    A Realtime item responding to an MCP approval request.
    - id: String
      
      The unique ID of the approval response.
    - approval_request_id: String
      
      The ID of the approval request being answered.
    - approve: bool
      
      Whether the request was approved.
    - type: :mcp_approval_response
      
      The type of the item. Always mcp_approval_response.
      - :mcp_approval_response
    - reason: String
      
      Optional reason for the decision.
  - class RealtimeMcpListTools
    
    A Realtime item listing tools available on an MCP server.
    - server_label: String
      
      The label of the MCP server.
    - tools: Array[Tool{ input_schema, name, annotations, description}]
      
      The tools available on the server.
      - input_schema: untyped
        
        The JSON schema describing the tool's input.
      - name: String
        
        The name of the tool.
      - annotations: untyped
        
        Additional annotations about the tool.
      - description: String
        
        The description of the tool.
    - type: :mcp_list_tools
      
      The type of the item. Always mcp_list_tools.
      - :mcp_list_tools
    - id: String
      
      The unique ID of the list.
  - class RealtimeMcpToolCall
    
    A Realtime item representing an invocation of a tool on an MCP server.
    - id: String
      
      The unique ID of the tool call.
    - arguments: String
      
      A JSON string of the arguments passed to the tool.
    - name: String
      
      The name of the tool that was run.
    - server_label: String
      
      The label of the MCP server running the tool.
    - type: :mcp_call
      
      The type of the item. Always mcp_call.
      - :mcp_call
    - approval_request_id: String
      
      The ID of an associated approval request, if any.
    - error: RealtimeMcpProtocolError | RealtimeMcpToolExecutionError | RealtimeMcphttpError
      
      The error from the tool call, if any.
      - class RealtimeMcpProtocolError
        
        code: Integer
        
        message: String
        
        type: :protocol_error
        
        :protocol_error
      - class RealtimeMcpToolExecutionError
        
        message: String
        
        type: :tool_execution_error
        
        :tool_execution_error
      - class RealtimeMcphttpError
        
        code: Integer
        
        message: String
        
        type: :http_error
        
        :http_error
    - output: String
      
      The output from the tool call.
  - class RealtimeMcpApprovalRequest
    
    A Realtime item requesting human approval of a tool invocation.
    - id: String
      
      The unique ID of the approval request.
    - arguments: String
      
      A JSON string of arguments for the tool.
    - name: String
      
      The name of the tool to run.
    - server_label: String
      
      The label of the MCP server making the request.
    - type: :mcp_approval_request
      
      The type of the item. Always mcp_approval_request.
      - :mcp_approval_request
- type: :"conversation.item.done"
  
  The event type, must be conversation.item.done.
  - :"conversation.item.done"
- previous_item_id: String
  
  The ID of the item that precedes this one, if any. This is used to maintain ordering when items are inserted.

Conversation Item Input Audio Transcription Completed Event

class ConversationItemInputAudioTranscriptionCompletedEvent

This event is the output of audio transcription for user audio written to the user audio buffer. Transcription begins when the input audio buffer is committed by the client or server (when VAD is enabled). Transcription runs asynchronously with Response creation, so this event may come before or after the Response events.

Realtime API models accept audio natively, and thus input transcription is a separate process run on a separate ASR (Automatic Speech Recognition) model. The transcript may diverge somewhat from the model's interpretation, and should be treated as a rough guide.
- content_index: Integer
  
  The index of the content part containing the audio.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the item containing the audio that is being transcribed.
- transcript: String
  
  The transcribed text.
- type: :"conversation.item.input_audio_transcription.completed"
  
  The event type, must be conversation.item.input_audio_transcription.completed.
  - :"conversation.item.input_audio_transcription.completed"
- usage: TranscriptTextUsageTokens{ input_tokens, output_tokens, total_tokens, 2 more} | TranscriptTextUsageDuration{ seconds, type}
  
  Usage statistics for the transcription, this is billed according to the ASR model's pricing rather than the realtime model's pricing.
  - class TranscriptTextUsageTokens
    
    Usage statistics for models billed by token usage.
    - input_tokens: Integer
      
      Number of input tokens billed for this request.
    - output_tokens: Integer
      
      Number of output tokens generated.
    - total_tokens: Integer
      
      Total number of tokens used (input + output).
    - type: :tokens
      
      The type of the usage object. Always tokens for this variant.
      - :tokens
    - input_token_details: InputTokenDetails{ audio_tokens, text_tokens}
      
      Details about the input tokens billed for this request.
      - audio_tokens: Integer
        
        Number of audio tokens billed for this request.
      - text_tokens: Integer
        
        Number of text tokens billed for this request.
  - class TranscriptTextUsageDuration
    
    Usage statistics for models billed by audio input duration.
    - seconds: Float
      
      Duration of the input audio in seconds.
    - type: :duration
      
      The type of the usage object. Always duration for this variant.
      - :duration
- logprobs: Array[LogProbProperties]
  
  The log probabilities of the transcription.
  - token: String
    
    The token that was used to generate the log probability.
  - bytes: Array[Integer]
    
    The bytes that were used to generate the log probability.
  - logprob: Float
    
    The log probability of the token.

Conversation Item Input Audio Transcription Delta Event

class ConversationItemInputAudioTranscriptionDeltaEvent

Returned when the text value of an input audio transcription content part is updated with incremental transcription results.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the item containing the audio that is being transcribed.
- type: :"conversation.item.input_audio_transcription.delta"
  
  The event type, must be conversation.item.input_audio_transcription.delta.
  - :"conversation.item.input_audio_transcription.delta"
- content_index: Integer
  
  The index of the content part in the item's content array.
- delta: String
  
  The text delta.
- logprobs: Array[LogProbProperties]
  
  The log probabilities of the transcription. These can be enabled by configurating the session with "include": ["item.input_audio_transcription.logprobs"]. Each entry in the array corresponds a log probability of which token would be selected for this chunk of transcription. This can help to identify if it was possible there were multiple valid options for a given chunk of transcription.
  - token: String
    
    The token that was used to generate the log probability.
  - bytes: Array[Integer]
    
    The bytes that were used to generate the log probability.
  - logprob: Float
    
    The log probability of the token.

Conversation Item Input Audio Transcription Failed Event

class ConversationItemInputAudioTranscriptionFailedEvent

Returned when input audio transcription is configured, and a transcription request for a user message failed. These events are separate from other error events so that the client can identify the related Item.
- content_index: Integer
  
  The index of the content part containing the audio.
- error: Error{ code, message, param, type}
  
  Details of the transcription error.
  - code: String
    
    Error code, if any.
  - message: String
    
    A human-readable error message.
  - param: String
    
    Parameter related to the error, if any.
  - type: String
    
    The type of error.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the user message item.
- type: :"conversation.item.input_audio_transcription.failed"
  
  The event type, must be conversation.item.input_audio_transcription.failed.
  - :"conversation.item.input_audio_transcription.failed"

Conversation Item Input Audio Transcription Segment

class ConversationItemInputAudioTranscriptionSegment

Returned when an input audio transcription segment is identified for an item.
- id: String
  
  The segment identifier.
- content_index: Integer
  
  The index of the input audio content part within the item.
- end_: Float
  
  End time of the segment in seconds.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the item containing the input audio content.
- speaker: String
  
  The detected speaker label for this segment.
- start: Float
  
  Start time of the segment in seconds.
- text: String
  
  The text for this segment.
- type: :"conversation.item.input_audio_transcription.segment"
  
  The event type, must be conversation.item.input_audio_transcription.segment.
  - :"conversation.item.input_audio_transcription.segment"

Conversation Item Retrieve Event

class ConversationItemRetrieveEvent

Send this event when you want to retrieve the server's representation of a specific item in the conversation history. This is useful, for example, to inspect user audio after noise cancellation and VAD. The server will respond with a conversation.item.retrieved event, unless the item does not exist in the conversation history, in which case the server will respond with an error.
- item_id: String
  
  The ID of the item to retrieve.
- type: :"conversation.item.retrieve"
  
  The event type, must be conversation.item.retrieve.
  - :"conversation.item.retrieve"
- event_id: String
  
  Optional client-generated ID used to identify this event.

Conversation Item Truncate Event

class ConversationItemTruncateEvent

Send this event to truncate a previous assistant message’s audio. The server will produce audio faster than realtime, so this event is useful when the user interrupts to truncate audio that has already been sent to the client but not yet played. This will synchronize the server's understanding of the audio with the client's playback.

Truncating audio will delete the server-side text transcript to ensure there is not text in the context that hasn't been heard by the user.

If successful, the server will respond with a conversation.item.truncated event.
- audio_end_ms: Integer
  
  Inclusive duration up to which audio is truncated, in milliseconds. If the audio_end_ms is greater than the actual audio duration, the server will respond with an error.
- content_index: Integer
  
  The index of the content part to truncate. Set this to 0.
- item_id: String
  
  The ID of the assistant message item to truncate. Only assistant message items can be truncated.
- type: :"conversation.item.truncate"
  
  The event type, must be conversation.item.truncate.
  - :"conversation.item.truncate"
- event_id: String
  
  Optional client-generated ID used to identify this event.

Conversation Item Truncated Event

class ConversationItemTruncatedEvent

Returned when an earlier assistant audio message item is truncated by the client with a conversation.item.truncate event. This event is used to synchronize the server's understanding of the audio with the client's playback.

This action will truncate the audio and remove the server-side text transcript to ensure there is no text in the context that hasn't been heard by the user.
- audio_end_ms: Integer
  
  The duration up to which the audio was truncated, in milliseconds.
- content_index: Integer
  
  The index of the content part that was truncated.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the assistant message item that was truncated.
- type: :"conversation.item.truncated"
  
  The event type, must be conversation.item.truncated.
  - :"conversation.item.truncated"

Conversation Item With Reference

class ConversationItemWithReference

The item to add to the conversation.
- id: String
  
  For an item of type (message | function_call | function_call_output) this field allows the client to assign the unique ID of the item. It is not required because the server will generate one if not provided.
  
  For an item of type item_reference, this field is required and is a reference to any item that has previously existed in the conversation.
- arguments: String
  
  The arguments of the function call (for function_call items).
- call_id: String
  
  The ID of the function call (for function_call and function_call_output items). If passed on a function_call_output item, the server will check that a function_call item with the same ID exists in the conversation history.
- content: Array[Content{ id, audio, text, 2 more}]
  
  The content of the message, applicable for message items.
  - Message items of role system support only input_text content
  - Message items of role user support input_text and input_audio content
  - Message items of role assistant support text content.
  - id: String
    
    ID of a previous conversation item to reference (for item_reference content types in response.create events). These can reference both client and server created items.
  - audio: String
    
    Base64-encoded audio bytes, used for input_audio content type.
  - text: String
    
    The text content, used for input_text and text content types.
  - transcript: String
    
    The transcript of the audio, used for input_audio content type.
  - type: :input_text | :input_audio | :item_reference | :text
    
    The content type (input_text, input_audio, item_reference, text).
    - :input_text
    - :input_audio
    - :item_reference
    - :text
- name: String
  
  The name of the function being called (for function_call items).
- object: :"realtime.item"
  
  Identifier for the API object being returned - always realtime.item.
  - :"realtime.item"
- output: String
  
  The output of the function call (for function_call_output items).
- role: :user | :assistant | :system
  
  The role of the message sender (user, assistant, system), only applicable for message items.
  - :user
  - :assistant
  - :system
- status: :completed | :incomplete | :in_progress
  
  The status of the item (completed, incomplete, in_progress). These have no effect on the conversation, but are accepted for consistency with the conversation.item.created event.
  - :completed
  - :incomplete
  - :in_progress
- type: :message | :function_call | :function_call_output | :item_reference
  
  The type of the item (message, function_call, function_call_output, item_reference).
  - :message
  - :function_call
  - :function_call_output
  - :item_reference

Input Audio Buffer Append Event

class InputAudioBufferAppendEvent

Send this event to append audio bytes to the input audio buffer. The audio buffer is temporary storage you can write to and later commit. A "commit" will create a new user message item in the conversation history from the buffer content and clear the buffer. Input audio transcription (if enabled) will be generated when the buffer is committed.

If VAD is enabled the audio buffer is used to detect speech and the server will decide when to commit. When Server VAD is disabled, you must commit the audio buffer manually. Input audio noise reduction operates on writes to the audio buffer.

The client may choose how much audio to place in each event up to a maximum of 15 MiB, for example streaming smaller chunks from the client may allow the VAD to be more responsive. Unlike most other client events, the server will not send a confirmation response to this event.
- audio: String
  
  Base64-encoded audio bytes. This must be in the format specified by the input_audio_format field in the session configuration.
- type: :"input_audio_buffer.append"
  
  The event type, must be input_audio_buffer.append.
  - :"input_audio_buffer.append"
- event_id: String
  
  Optional client-generated ID used to identify this event.

Input Audio Buffer Clear Event

class InputAudioBufferClearEvent

Send this event to clear the audio bytes in the buffer. The server will respond with an input_audio_buffer.cleared event.
- type: :"input_audio_buffer.clear"
  
  The event type, must be input_audio_buffer.clear.
  - :"input_audio_buffer.clear"
- event_id: String
  
  Optional client-generated ID used to identify this event.

Input Audio Buffer Cleared Event

class InputAudioBufferClearedEvent

Returned when the input audio buffer is cleared by the client with a input_audio_buffer.clear event.
- event_id: String
  
  The unique ID of the server event.
- type: :"input_audio_buffer.cleared"
  
  The event type, must be input_audio_buffer.cleared.
  - :"input_audio_buffer.cleared"

Input Audio Buffer Commit Event

class InputAudioBufferCommitEvent

Send this event to commit the user input audio buffer, which will create a new user message item in the conversation. This event will produce an error if the input audio buffer is empty. When in Server VAD mode, the client does not need to send this event, the server will commit the audio buffer automatically.

Committing the input audio buffer will trigger input audio transcription (if enabled in session configuration), but it will not create a response from the model. The server will respond with an input_audio_buffer.committed event.
- type: :"input_audio_buffer.commit"
  
  The event type, must be input_audio_buffer.commit.
  - :"input_audio_buffer.commit"
- event_id: String
  
  Optional client-generated ID used to identify this event.

Input Audio Buffer Committed Event

class InputAudioBufferCommittedEvent

Returned when an input audio buffer is committed, either by the client or automatically in server VAD mode. The item_id property is the ID of the user message item that will be created, thus a conversation.item.created event will also be sent to the client.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the user message item that will be created.
- type: :"input_audio_buffer.committed"
  
  The event type, must be input_audio_buffer.committed.
  - :"input_audio_buffer.committed"
- previous_item_id: String
  
  The ID of the preceding item after which the new item will be inserted. Can be null if the item has no predecessor.

Input Audio Buffer Dtmf Event Received Event

class InputAudioBufferDtmfEventReceivedEvent

SIP Only: Returned when an DTMF event is received. A DTMF event is a message that represents a telephone keypad press (0–9, *, #, A–D). The event property is the keypad that the user press. The received_at is the UTC Unix Timestamp that the server received the event.
- event: String
  
  The telephone keypad that was pressed by the user.
- received_at: Integer
  
  UTC Unix Timestamp when DTMF Event was received by server.
- type: :"input_audio_buffer.dtmf_event_received"
  
  The event type, must be input_audio_buffer.dtmf_event_received.
  - :"input_audio_buffer.dtmf_event_received"

Input Audio Buffer Speech Started Event

class InputAudioBufferSpeechStartedEvent

Sent by the server when in server_vad mode to indicate that speech has been detected in the audio buffer. This can happen any time audio is added to the buffer (unless speech is already detected). The client may want to use this event to interrupt audio playback or provide visual feedback to the user.

The client should expect to receive a input_audio_buffer.speech_stopped event when speech stops. The item_id property is the ID of the user message item that will be created when speech stops and will also be included in the input_audio_buffer.speech_stopped event (unless the client manually commits the audio buffer during VAD activation).
- audio_start_ms: Integer
  
  Milliseconds from the start of all audio written to the buffer during the session when speech was first detected. This will correspond to the beginning of audio sent to the model, and thus includes the prefix_padding_ms configured in the Session.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the user message item that will be created when speech stops.
- type: :"input_audio_buffer.speech_started"
  
  The event type, must be input_audio_buffer.speech_started.
  - :"input_audio_buffer.speech_started"

Input Audio Buffer Speech Stopped Event

class InputAudioBufferSpeechStoppedEvent

Returned in server_vad mode when the server detects the end of speech in the audio buffer. The server will also send an conversation.item.created event with the user message item that is created from the audio buffer.
- audio_end_ms: Integer
  
  Milliseconds since the session started when speech stopped. This will correspond to the end of audio sent to the model, and thus includes the min_silence_duration_ms configured in the Session.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the user message item that will be created.
- type: :"input_audio_buffer.speech_stopped"
  
  The event type, must be input_audio_buffer.speech_stopped.
  - :"input_audio_buffer.speech_stopped"

Input Audio Buffer Timeout Triggered

class InputAudioBufferTimeoutTriggered

Returned when the Server VAD timeout is triggered for the input audio buffer. This is configured with idle_timeout_ms in the turn_detection settings of the session, and it indicates that there hasn't been any speech detected for the configured duration.

The audio_start_ms and audio_end_ms fields indicate the segment of audio after the last model response up to the triggering time, as an offset from the beginning of audio written to the input audio buffer. This means it demarcates the segment of audio that was silent and the difference between the start and end values will roughly match the configured timeout.

The empty audio will be committed to the conversation as an input_audio item (there will be a input_audio_buffer.committed event) and a model response will be generated. There may be speech that didn't trigger VAD but is still detected by the model, so the model may respond with something relevant to the conversation or a prompt to continue speaking.
- audio_end_ms: Integer
  
  Millisecond offset of audio written to the input audio buffer at the time the timeout was triggered.
- audio_start_ms: Integer
  
  Millisecond offset of audio written to the input audio buffer that was after the playback time of the last model response.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the item associated with this segment.
- type: :"input_audio_buffer.timeout_triggered"
  
  The event type, must be input_audio_buffer.timeout_triggered.
  - :"input_audio_buffer.timeout_triggered"

Log Prob Properties

class LogProbProperties

A log probability object.
- token: String
  
  The token that was used to generate the log probability.
- bytes: Array[Integer]
  
  The bytes that were used to generate the log probability.
- logprob: Float
  
  The log probability of the token.

Mcp List Tools Completed

class McpListToolsCompleted

Returned when listing MCP tools has completed for an item.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the MCP list tools item.
- type: :"mcp_list_tools.completed"
  
  The event type, must be mcp_list_tools.completed.
  - :"mcp_list_tools.completed"

Mcp List Tools Failed

class McpListToolsFailed

Returned when listing MCP tools has failed for an item.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the MCP list tools item.
- type: :"mcp_list_tools.failed"
  
  The event type, must be mcp_list_tools.failed.
  - :"mcp_list_tools.failed"

Mcp List Tools In Progress

class McpListToolsInProgress

Returned when listing MCP tools is in progress for an item.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the MCP list tools item.
- type: :"mcp_list_tools.in_progress"
  
  The event type, must be mcp_list_tools.in_progress.
  - :"mcp_list_tools.in_progress"

Noise Reduction Type

NoiseReductionType = :near_field | :far_field

Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
- :near_field
- :far_field

Output Audio Buffer Clear Event

class OutputAudioBufferClearEvent

WebRTC/SIP Only: Emit to cut off the current audio response. This will trigger the server to stop generating audio and emit a output_audio_buffer.cleared event. This event should be preceded by a response.cancel client event to stop the generation of the current response. Learn more.
- type: :"output_audio_buffer.clear"
  
  The event type, must be output_audio_buffer.clear.
  - :"output_audio_buffer.clear"
- event_id: String
  
  The unique ID of the client event used for error handling.

Rate Limits Updated Event

class RateLimitsUpdatedEvent

Emitted at the beginning of a Response to indicate the updated rate limits. When a Response is created some tokens will be "reserved" for the output tokens, the rate limits shown here reflect that reservation, which is then adjusted accordingly once the Response is completed.
- event_id: String
  
  The unique ID of the server event.
- rate_limits: Array[RateLimit{ limit, name, remaining, reset_seconds}]
  
  List of rate limit information.
  - limit: Integer
    
    The maximum allowed value for the rate limit.
  - name: :requests | :tokens
    
    The name of the rate limit (requests, tokens).
    - :requests
    - :tokens
  - remaining: Integer
    
    The remaining value before the limit is reached.
  - reset_seconds: Float
    
    Seconds until the rate limit resets.
- type: :"rate_limits.updated"
  
  The event type, must be rate_limits.updated.
  - :"rate_limits.updated"

Realtime Audio Config

class RealtimeAudioConfig

Configuration for input and output audio.
- input: RealtimeAudioConfigInput
  - format_: RealtimeAudioFormats
    
    The format of the input audio.
    - class AudioPCM
      
      The PCM audio format. Only a 24kHz sample rate is supported.
      - rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
      - type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
    - class AudioPCMU
      
      The G.711 μ-law format.
      - type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
    - class AudioPCMA
      
      The G.711 A-law format.
      - type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
  - noise_reduction: NoiseReduction{ type}
    
    Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
    - type: NoiseReductionType
      
      Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
      - :near_field
      - :far_field
  - transcription: AudioTranscription
    
    Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
    - delay: :minimal | :low | :medium | 2 more
      
      Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
      - :minimal
      - :low
      - :medium
      - :high
      - :xhigh
    - language: String
      
      The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
    - model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
      
      The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
      - String = String
      - Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        :"whisper-1"
        
        :"gpt-4o-mini-transcribe"
        
        :"gpt-4o-mini-transcribe-2025-12-15"
        
        :"gpt-4o-transcribe"
        
        :"gpt-4o-transcribe-diarize"
        
        :"gpt-realtime-whisper"
    - prompt: String
      
      An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
  - turn_detection: RealtimeAudioInputTurnDetection
    
    Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
    
    Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
    
    Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
    
    For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
    - class ServerVad
      
      Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
      - type: :server_vad
        
        Type of turn detection, server_vad to turn on simple Server VAD.
        
        :server_vad
      - create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
      - idle_timeout_ms: Integer
        
        Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
        
        The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
        
        An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
      - interrupt_response: bool
        
        Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
      - prefix_padding_ms: Integer
        
        Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
      - silence_duration_ms: Integer
        
        Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
      - threshold: Float
        
        Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
    - class SemanticVad
      
      Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
      - type: :semantic_vad
        
        Type of turn detection, semantic_vad to turn on Semantic VAD.
        
        :semantic_vad
      - create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs.
      - eagerness: :low | :medium | :high | :auto
        
        Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
        
        :low
        
        :medium
        
        :high
        
        :auto
      - interrupt_response: bool
        
        Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.
- output: RealtimeAudioConfigOutput
  - format_: RealtimeAudioFormats
    
    The format of the output audio.
  - speed: Float
    
    The speed of the model's spoken response as a multiple of the original speed. 1.0 is the default speed. 0.25 is the minimum speed. 1.5 is the maximum speed. This value can only be changed in between model turns, not while a response is in progress.
    
    This parameter is a post-processing adjustment to the audio after it is generated, it's also possible to prompt the model to speak faster or slower.
  - voice: String | :alloy | :ash | :ballad | 7 more | ID{ id}
    
    The voice the model uses to respond. Supported built-in voices are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. You may also provide a custom voice object with an id, for example { "id": "voice_1234" }. Voice cannot be changed during the session once the model has responded with audio at least once. We recommend marin and cedar for best quality.
    - String = String
    - Voice = :alloy | :ash | :ballad | 7 more
      - :alloy
      - :ash
      - :ballad
      - :coral
      - :echo
      - :sage
      - :shimmer
      - :verse
      - :marin
      - :cedar
    - class ID
      
      Custom voice reference.
      - id: String
        
        The custom voice ID, e.g. voice_1234.

Realtime Audio Config Input

class RealtimeAudioConfigInput
- format_: RealtimeAudioFormats
  
  The format of the input audio.
  - class AudioPCM
    
    The PCM audio format. Only a 24kHz sample rate is supported.
    - rate: 24000
      
      The sample rate of the audio. Always 24000.
      - 24000
    - type: :"audio/pcm"
      
      The audio format. Always audio/pcm.
      - :"audio/pcm"
  - class AudioPCMU
    
    The G.711 μ-law format.
    - type: :"audio/pcmu"
      
      The audio format. Always audio/pcmu.
      - :"audio/pcmu"
  - class AudioPCMA
    
    The G.711 A-law format.
    - type: :"audio/pcma"
      
      The audio format. Always audio/pcma.
      - :"audio/pcma"
- noise_reduction: NoiseReduction{ type}
  
  Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
  - type: NoiseReductionType
    
    Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
    - :near_field
    - :far_field
- transcription: AudioTranscription
  
  Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
  - delay: :minimal | :low | :medium | 2 more
    
    Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
    - :minimal
    - :low
    - :medium
    - :high
    - :xhigh
  - language: String
    
    The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
  - model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
    
    The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
    - String = String
    - Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
      
      The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
      - :"whisper-1"
      - :"gpt-4o-mini-transcribe"
      - :"gpt-4o-mini-transcribe-2025-12-15"
      - :"gpt-4o-transcribe"
      - :"gpt-4o-transcribe-diarize"
      - :"gpt-realtime-whisper"
  - prompt: String
    
    An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
- turn_detection: RealtimeAudioInputTurnDetection
  
  Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
  
  Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
  
  Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
  
  For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
  - class ServerVad
    
    Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
    - type: :server_vad
      
      Type of turn detection, server_vad to turn on simple Server VAD.
      - :server_vad
    - create_response: bool
      
      Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
      
      If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
    - idle_timeout_ms: Integer
      
      Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
      
      The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
      
      An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
    - interrupt_response: bool
      
      Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
      
      If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
    - prefix_padding_ms: Integer
      
      Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
    - silence_duration_ms: Integer
      
      Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
    - threshold: Float
      
      Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
  - class SemanticVad
    
    Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
    - type: :semantic_vad
      
      Type of turn detection, semantic_vad to turn on Semantic VAD.
      - :semantic_vad
    - create_response: bool
      
      Whether or not to automatically generate a response when a VAD stop event occurs.
    - eagerness: :low | :medium | :high | :auto
      
      Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
      - :low
      - :medium
      - :high
      - :auto
    - interrupt_response: bool
      
      Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.

Realtime Audio Config Output

class RealtimeAudioConfigOutput
- format_: RealtimeAudioFormats
  
  The format of the output audio.
  - class AudioPCM
    
    The PCM audio format. Only a 24kHz sample rate is supported.
    - rate: 24000
      
      The sample rate of the audio. Always 24000.
      - 24000
    - type: :"audio/pcm"
      
      The audio format. Always audio/pcm.
      - :"audio/pcm"
  - class AudioPCMU
    
    The G.711 μ-law format.
    - type: :"audio/pcmu"
      
      The audio format. Always audio/pcmu.
      - :"audio/pcmu"
  - class AudioPCMA
    
    The G.711 A-law format.
    - type: :"audio/pcma"
      
      The audio format. Always audio/pcma.
      - :"audio/pcma"
- speed: Float
  
  The speed of the model's spoken response as a multiple of the original speed. 1.0 is the default speed. 0.25 is the minimum speed. 1.5 is the maximum speed. This value can only be changed in between model turns, not while a response is in progress.
  
  This parameter is a post-processing adjustment to the audio after it is generated, it's also possible to prompt the model to speak faster or slower.
- voice: String | :alloy | :ash | :ballad | 7 more | ID{ id}
  
  The voice the model uses to respond. Supported built-in voices are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. You may also provide a custom voice object with an id, for example { "id": "voice_1234" }. Voice cannot be changed during the session once the model has responded with audio at least once. We recommend marin and cedar for best quality.
  - String = String
  - Voice = :alloy | :ash | :ballad | 7 more
    - :alloy
    - :ash
    - :ballad
    - :coral
    - :echo
    - :sage
    - :shimmer
    - :verse
    - :marin
    - :cedar
  - class ID
    
    Custom voice reference.
    - id: String
      
      The custom voice ID, e.g. voice_1234.

Realtime Audio Formats

RealtimeAudioFormats = AudioPCM{ rate, type} | AudioPCMU{ type} | AudioPCMA{ type}

The PCM audio format. Only a 24kHz sample rate is supported.
- class AudioPCM
  
  The PCM audio format. Only a 24kHz sample rate is supported.
  - rate: 24000
    
    The sample rate of the audio. Always 24000.
    - 24000
  - type: :"audio/pcm"
    
    The audio format. Always audio/pcm.
    - :"audio/pcm"
- class AudioPCMU
  
  The G.711 μ-law format.
  - type: :"audio/pcmu"
    
    The audio format. Always audio/pcmu.
    - :"audio/pcmu"
- class AudioPCMA
  
  The G.711 A-law format.
  - type: :"audio/pcma"
    
    The audio format. Always audio/pcma.
    - :"audio/pcma"

Realtime Audio Input Turn Detection

RealtimeAudioInputTurnDetection = ServerVad{ type, create_response, idle_timeout_ms, 4 more} | SemanticVad{ type, create_response, eagerness, interrupt_response}

Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.

Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.

Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.

For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
- class ServerVad
  
  Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
  - type: :server_vad
    
    Type of turn detection, server_vad to turn on simple Server VAD.
    - :server_vad
  - create_response: bool
    
    Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
    
    If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
  - idle_timeout_ms: Integer
    
    Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
    
    The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
    
    An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
  - interrupt_response: bool
    
    Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
    
    If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
  - prefix_padding_ms: Integer
    
    Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
  - silence_duration_ms: Integer
    
    Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
  - threshold: Float
    
    Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
- class SemanticVad
  
  Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
  - type: :semantic_vad
    
    Type of turn detection, semantic_vad to turn on Semantic VAD.
    - :semantic_vad
  - create_response: bool
    
    Whether or not to automatically generate a response when a VAD stop event occurs.
  - eagerness: :low | :medium | :high | :auto
    
    Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
    - :low
    - :medium
    - :high
    - :auto
  - interrupt_response: bool
    
    Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.

Realtime Client Event

RealtimeClientEvent = ConversationItemCreateEvent | ConversationItemDeleteEvent | ConversationItemRetrieveEvent | 8 more

A realtime client event.
- class ConversationItemCreateEvent
  
  Add a new Item to the Conversation's context, including messages, function calls, and function call responses. This event can be used both to populate a "history" of the conversation and to add new items mid-stream, but has the current limitation that it cannot populate assistant audio messages.
  
  If successful, the server will respond with a conversation.item.created event, otherwise an error event will be sent.
  - item: ConversationItem
    
    A single item within a Realtime conversation.
    - class RealtimeConversationItemSystemMessage
      
      A system message in a Realtime conversation can be used to provide additional context or instructions to the model. This is similar but distinct from the instruction prompt provided at the start of a conversation, as system messages can be added at any point in the conversation. For major changes to the conversation's behavior, use instructions, but for smaller updates (e.g. "the user is now asking about a different topic"), use system messages.
      - content: Array[Content{ text, type}]
        
        The content of the message.
        
        text: String
        
        The text content.
        
        type: :input_text
        
        The content type. Always input_text for system messages.
        
        :input_text
      - role: :system
        
        The role of the message sender. Always system.
        
        :system
      - type: :message
        
        The type of the item. Always message.
        
        :message
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeConversationItemUserMessage
      
      A user message item in a Realtime conversation.
      - content: Array[Content{ audio, detail, image_url, 3 more}]
        
        The content of the message.
        
        audio: String
        
        Base64-encoded audio bytes (for input_audio), these will be parsed as the format specified in the session input audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
        
        detail: :auto | :low | :high
        
        The detail level of the image (for input_image). auto will default to high.
        
        :auto
        
        :low
        
        :high
        
        image_url: String
        
        Base64-encoded image bytes (for input_image) as a data URI. For example data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA.... Supported formats are PNG and JPEG.
        
        text: String
        
        The text content (for input_text).
        
        transcript: String
        
        Transcript of the audio (for input_audio). This is not sent to the model, but will be attached to the message item for reference.
        
        type: :input_text | :input_audio | :input_image
        
        The content type (input_text, input_audio, or input_image).
        
        :input_text
        
        :input_audio
        
        :input_image
      - role: :user
        
        The role of the message sender. Always user.
        
        :user
      - type: :message
        
        The type of the item. Always message.
        
        :message
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeConversationItemAssistantMessage
      
      An assistant message item in a Realtime conversation.
      - content: Array[Content{ audio, text, transcript, type}]
        
        The content of the message.
        
        audio: String
        
        Base64-encoded audio bytes, these will be parsed as the format specified in the session output audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
        
        text: String
        
        The text content.
        
        transcript: String
        
        The transcript of the audio content, this will always be present if the output type is audio.
        
        type: :output_text | :output_audio
        
        The content type, output_text or output_audio depending on the session output_modalities configuration.
        
        :output_text
        
        :output_audio
      - role: :assistant
        
        The role of the message sender. Always assistant.
        
        :assistant
      - type: :message
        
        The type of the item. Always message.
        
        :message
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeConversationItemFunctionCall
      
      A function call item in a Realtime conversation.
      - arguments: String
        
        The arguments of the function call. This is a JSON-encoded string representing the arguments passed to the function, for example {"arg1": "value1", "arg2": 42}.
      - name: String
        
        The name of the function being called.
      - type: :function_call
        
        The type of the item. Always function_call.
        
        :function_call
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - call_id: String
        
        The ID of the function call.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeConversationItemFunctionCallOutput
      
      A function call output item in a Realtime conversation.
      - call_id: String
        
        The ID of the function call this output is for.
      - output: String
        
        The output of the function call, this is free text and can contain any information or simply be empty.
      - type: :function_call_output
        
        The type of the item. Always function_call_output.
        
        :function_call_output
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeMcpApprovalResponse
      
      A Realtime item responding to an MCP approval request.
      - id: String
        
        The unique ID of the approval response.
      - approval_request_id: String
        
        The ID of the approval request being answered.
      - approve: bool
        
        Whether the request was approved.
      - type: :mcp_approval_response
        
        The type of the item. Always mcp_approval_response.
        
        :mcp_approval_response
      - reason: String
        
        Optional reason for the decision.
    - class RealtimeMcpListTools
      
      A Realtime item listing tools available on an MCP server.
      - server_label: String
        
        The label of the MCP server.
      - tools: Array[Tool{ input_schema, name, annotations, description}]
        
        The tools available on the server.
        
        input_schema: untyped
        
        The JSON schema describing the tool's input.
        
        name: String
        
        The name of the tool.
        
        annotations: untyped
        
        Additional annotations about the tool.
        
        description: String
        
        The description of the tool.
      - type: :mcp_list_tools
        
        The type of the item. Always mcp_list_tools.
        
        :mcp_list_tools
      - id: String
        
        The unique ID of the list.
    - class RealtimeMcpToolCall
      
      A Realtime item representing an invocation of a tool on an MCP server.
      - id: String
        
        The unique ID of the tool call.
      - arguments: String
        
        A JSON string of the arguments passed to the tool.
      - name: String
        
        The name of the tool that was run.
      - server_label: String
        
        The label of the MCP server running the tool.
      - type: :mcp_call
        
        The type of the item. Always mcp_call.
        
        :mcp_call
      - approval_request_id: String
        
        The ID of an associated approval request, if any.
      - error: RealtimeMcpProtocolError | RealtimeMcpToolExecutionError | RealtimeMcphttpError
        
        The error from the tool call, if any.
        
        class RealtimeMcpProtocolError
        
        code: Integer
        
        message: String
        
        type: :protocol_error
        
        :protocol_error
        
        class RealtimeMcpToolExecutionError
        
        message: String
        
        type: :tool_execution_error
        
        :tool_execution_error
        
        class RealtimeMcphttpError
        
        code: Integer
        
        message: String
        
        type: :http_error
        
        :http_error
      - output: String
        
        The output from the tool call.
    - class RealtimeMcpApprovalRequest
      
      A Realtime item requesting human approval of a tool invocation.
      - id: String
        
        The unique ID of the approval request.
      - arguments: String
        
        A JSON string of arguments for the tool.
      - name: String
        
        The name of the tool to run.
      - server_label: String
        
        The label of the MCP server making the request.
      - type: :mcp_approval_request
        
        The type of the item. Always mcp_approval_request.
        
        :mcp_approval_request
  - type: :"conversation.item.create"
    
    The event type, must be conversation.item.create.
    - :"conversation.item.create"
  - event_id: String
    
    Optional client-generated ID used to identify this event.
  - previous_item_id: String
    
    The ID of the preceding item after which the new item will be inserted. If not set, the new item will be appended to the end of the conversation.
    
    If set to root, the new item will be added to the beginning of the conversation.
    
    If set to an existing ID, it allows an item to be inserted mid-conversation. If the ID cannot be found, an error will be returned and the item will not be added.
- class ConversationItemDeleteEvent
  
  Send this event when you want to remove any item from the conversation history. The server will respond with a conversation.item.deleted event, unless the item does not exist in the conversation history, in which case the server will respond with an error.
  - item_id: String
    
    The ID of the item to delete.
  - type: :"conversation.item.delete"
    
    The event type, must be conversation.item.delete.
    - :"conversation.item.delete"
  - event_id: String
    
    Optional client-generated ID used to identify this event.
- class ConversationItemRetrieveEvent
  
  Send this event when you want to retrieve the server's representation of a specific item in the conversation history. This is useful, for example, to inspect user audio after noise cancellation and VAD. The server will respond with a conversation.item.retrieved event, unless the item does not exist in the conversation history, in which case the server will respond with an error.
  - item_id: String
    
    The ID of the item to retrieve.
  - type: :"conversation.item.retrieve"
    
    The event type, must be conversation.item.retrieve.
    - :"conversation.item.retrieve"
  - event_id: String
    
    Optional client-generated ID used to identify this event.
- class ConversationItemTruncateEvent
  
  Send this event to truncate a previous assistant message’s audio. The server will produce audio faster than realtime, so this event is useful when the user interrupts to truncate audio that has already been sent to the client but not yet played. This will synchronize the server's understanding of the audio with the client's playback.
  
  Truncating audio will delete the server-side text transcript to ensure there is not text in the context that hasn't been heard by the user.
  
  If successful, the server will respond with a conversation.item.truncated event.
  - audio_end_ms: Integer
    
    Inclusive duration up to which audio is truncated, in milliseconds. If the audio_end_ms is greater than the actual audio duration, the server will respond with an error.
  - content_index: Integer
    
    The index of the content part to truncate. Set this to 0.
  - item_id: String
    
    The ID of the assistant message item to truncate. Only assistant message items can be truncated.
  - type: :"conversation.item.truncate"
    
    The event type, must be conversation.item.truncate.
    - :"conversation.item.truncate"
  - event_id: String
    
    Optional client-generated ID used to identify this event.
- class InputAudioBufferAppendEvent
  
  Send this event to append audio bytes to the input audio buffer. The audio buffer is temporary storage you can write to and later commit. A "commit" will create a new user message item in the conversation history from the buffer content and clear the buffer. Input audio transcription (if enabled) will be generated when the buffer is committed.
  
  If VAD is enabled the audio buffer is used to detect speech and the server will decide when to commit. When Server VAD is disabled, you must commit the audio buffer manually. Input audio noise reduction operates on writes to the audio buffer.
  
  The client may choose how much audio to place in each event up to a maximum of 15 MiB, for example streaming smaller chunks from the client may allow the VAD to be more responsive. Unlike most other client events, the server will not send a confirmation response to this event.
  - audio: String
    
    Base64-encoded audio bytes. This must be in the format specified by the input_audio_format field in the session configuration.
  - type: :"input_audio_buffer.append"
    
    The event type, must be input_audio_buffer.append.
    - :"input_audio_buffer.append"
  - event_id: String
    
    Optional client-generated ID used to identify this event.
- class InputAudioBufferClearEvent
  
  Send this event to clear the audio bytes in the buffer. The server will respond with an input_audio_buffer.cleared event.
  - type: :"input_audio_buffer.clear"
    
    The event type, must be input_audio_buffer.clear.
    - :"input_audio_buffer.clear"
  - event_id: String
    
    Optional client-generated ID used to identify this event.
- class OutputAudioBufferClearEvent
  
  WebRTC/SIP Only: Emit to cut off the current audio response. This will trigger the server to stop generating audio and emit a output_audio_buffer.cleared event. This event should be preceded by a response.cancel client event to stop the generation of the current response. Learn more.
  - type: :"output_audio_buffer.clear"
    
    The event type, must be output_audio_buffer.clear.
    - :"output_audio_buffer.clear"
  - event_id: String
    
    The unique ID of the client event used for error handling.
- class InputAudioBufferCommitEvent
  
  Send this event to commit the user input audio buffer, which will create a new user message item in the conversation. This event will produce an error if the input audio buffer is empty. When in Server VAD mode, the client does not need to send this event, the server will commit the audio buffer automatically.
  
  Committing the input audio buffer will trigger input audio transcription (if enabled in session configuration), but it will not create a response from the model. The server will respond with an input_audio_buffer.committed event.
  - type: :"input_audio_buffer.commit"
    
    The event type, must be input_audio_buffer.commit.
    - :"input_audio_buffer.commit"
  - event_id: String
    
    Optional client-generated ID used to identify this event.
- class ResponseCancelEvent
  
  Send this event to cancel an in-progress response. The server will respond with a response.done event with a status of response.status=cancelled. If there is no response to cancel, the server will respond with an error. It's safe to call response.cancel even if no response is in progress, an error will be returned the session will remain unaffected.
  - type: :"response.cancel"
    
    The event type, must be response.cancel.
    - :"response.cancel"
  - event_id: String
    
    Optional client-generated ID used to identify this event.
  - response_id: String
    
    A specific response ID to cancel - if not provided, will cancel an in-progress response in the default conversation.
- class ResponseCreateEvent
  
  This event instructs the server to create a Response, which means triggering model inference. When in Server VAD mode, the server will create Responses automatically.
  
  A Response will include at least one Item, and may have two, in which case the second will be a function call. These Items will be appended to the conversation history by default.
  
  The server will respond with a response.created event, events for Items and content created, and finally a response.done event to indicate the Response is complete.
  
  The response.create event includes inference configuration like instructions and tools. If these are set, they will override the Session's configuration for this Response only.
  
  Responses can be created out-of-band of the default Conversation, meaning that they can have arbitrary input, and it's possible to disable writing the output to the Conversation. Only one Response can write to the default Conversation at a time, but otherwise multiple Responses can be created in parallel. The metadata field is a good way to disambiguate multiple simultaneous Responses.
  
  Clients can set conversation to none to create a Response that does not write to the default Conversation. Arbitrary input can be provided with the input field, which is an array accepting raw Items and references to existing Items.
  - type: :"response.create"
    
    The event type, must be response.create.
    - :"response.create"
  - event_id: String
    
    Optional client-generated ID used to identify this event.
  - response: RealtimeResponseCreateParams
    
    Create a new Realtime response with these parameters
    - audio: RealtimeResponseCreateAudioOutput
      
      Configuration for audio input and output.
      - output: Output{ format_, voice}
        
        format_: RealtimeAudioFormats
        
        The format of the output audio.
        
        class AudioPCM
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
        
        type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
        
        class AudioPCMU
        
        The G.711 μ-law format.
        
        type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
        
        class AudioPCMA
        
        The G.711 A-law format.
        
        type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
        
        voice: String | :alloy | :ash | :ballad | 7 more | ID{ id}
        
        The voice the model uses to respond. Supported built-in voices are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. You may also provide a custom voice object with an id, for example { "id": "voice_1234" }. Voice cannot be changed during the session once the model has responded with audio at least once. We recommend marin and cedar for best quality.
        
        String = String
        
        Voice = :alloy | :ash | :ballad | 7 more
        
        :alloy
        
        :ash
        
        :ballad
        
        :coral
        
        :echo
        
        :sage
        
        :shimmer
        
        :verse
        
        :marin
        
        :cedar
        
        class ID
        
        Custom voice reference.
        
        id: String
        
        The custom voice ID, e.g. voice_1234.
    - conversation: String | :auto | :none
      
      Controls which conversation the response is added to. Currently supports auto and none, with auto as the default value. The auto value means that the contents of the response will be added to the default conversation. Set this to none to create an out-of-band response which will not add items to default conversation.
      - String = String
      - Conversation = :auto | :none
        
        Controls which conversation the response is added to. Currently supports auto and none, with auto as the default value. The auto value means that the contents of the response will be added to the default conversation. Set this to none to create an out-of-band response which will not add items to default conversation.
        
        :auto
        
        :none
    - input: Array[ConversationItem]
      
      Input items to include in the prompt for the model. Using this field creates a new context for this Response instead of using the default conversation. An empty array [] will clear the context for this Response. Note that this can include references to items that previously appeared in the session using their id.
      - class RealtimeConversationItemSystemMessage
        
        A system message in a Realtime conversation can be used to provide additional context or instructions to the model. This is similar but distinct from the instruction prompt provided at the start of a conversation, as system messages can be added at any point in the conversation. For major changes to the conversation's behavior, use instructions, but for smaller updates (e.g. "the user is now asking about a different topic"), use system messages.
      - class RealtimeConversationItemUserMessage
        
        A user message item in a Realtime conversation.
      - class RealtimeConversationItemAssistantMessage
        
        An assistant message item in a Realtime conversation.
      - class RealtimeConversationItemFunctionCall
        
        A function call item in a Realtime conversation.
      - class RealtimeConversationItemFunctionCallOutput
        
        A function call output item in a Realtime conversation.
      - class RealtimeMcpApprovalResponse
        
        A Realtime item responding to an MCP approval request.
      - class RealtimeMcpListTools
        
        A Realtime item listing tools available on an MCP server.
      - class RealtimeMcpToolCall
        
        A Realtime item representing an invocation of a tool on an MCP server.
      - class RealtimeMcpApprovalRequest
        
        A Realtime item requesting human approval of a tool invocation.
    - instructions: String
      
      The default system instructions (i.e. system message) prepended to model calls. This field allows the client to guide the model on desired responses. The model can be instructed on response content and format, (e.g. "be extremely succinct", "act friendly", "here are examples of good responses") and on audio behavior (e.g. "talk quickly", "inject emotion into your voice", "laugh frequently"). The instructions are not guaranteed to be followed by the model, but they provide guidance to the model on the desired behavior. Note that the server sets default instructions which will be used if this field is not set and are visible in the session.created event at the start of the session.
    - max_output_tokens: Integer | :inf
      
      Maximum number of output tokens for a single assistant response, inclusive of tool calls. Provide an integer between 1 and 4096 to limit output tokens, or inf for the maximum available tokens for a given model. Defaults to inf.
      - Integer = Integer
      - MaxOutputTokens = :inf
        
        :inf
    - metadata: Metadata
      
      Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard.
      
      Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.
    - output_modalities: Array[:text | :audio]
      
      The set of modalities the model used to respond, currently the only possible values are [\"audio\"], [\"text\"]. Audio output always include a text transcript. Setting the output to mode text will disable audio output from the model.
      - :text
      - :audio
    - parallel_tool_calls: bool
      
      Whether the model may call multiple tools in parallel. Only supported by reasoning Realtime models such as gpt-realtime-2.
    - prompt: ResponsePrompt
      
      Reference to a prompt template and its variables. Learn more.
      - id: String
        
        The unique identifier of the prompt template to use.
      - variables: Hash[Symbol, String | ResponseInputText | ResponseInputImage | ResponseInputFile]
        
        Optional map of values to substitute in for variables in your prompt. The substitution values can either be strings, or other Response input types like images or files.
        
        String = String
        
        class ResponseInputText
        
        A text input to the model.
        
        text: String
        
        The text input to the model.
        
        type: :input_text
        
        The type of the input item. Always input_text.
        
        :input_text
        
        class ResponseInputImage
        
        An image input to the model. Learn about image inputs.
        
        detail: :low | :high | :auto | :original
        
        The detail level of the image to be sent to the model. One of high, low, auto, or original. Defaults to auto.
        
        :low
        
        :high
        
        :auto
        
        :original
        
        type: :input_image
        
        The type of the input item. Always input_image.
        
        :input_image
        
        file_id: String
        
        The ID of the file to be sent to the model.
        
        image_url: String
        
        The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.
        
        class ResponseInputFile
        
        A file input to the model.
        
        type: :input_file
        
        The type of the input item. Always input_file.
        
        :input_file
        
        detail: :low | :high
        
        The detail level of the file to be sent to the model. Use low for the default rendering behavior, or high to render the file at higher quality. Defaults to low.
        
        :low
        
        :high
        
        file_data: String
        
        The content of the file to be sent to the model.
        
        file_id: String
        
        The ID of the file to be sent to the model.
        
        file_url: String
        
        The URL of the file to be sent to the model.
        
        filename: String
        
        The name of the file to be sent to the model.
      - version: String
        
        Optional version of the prompt template.
    - reasoning: RealtimeReasoning
      
      Configuration for reasoning-capable Realtime models such as gpt-realtime-2.
      - effort: RealtimeReasoningEffort
        
        Constrains effort on reasoning for reasoning-capable Realtime models such as gpt-realtime-2.
        
        :minimal
        
        :low
        
        :medium
        
        :high
        
        :xhigh
    - tool_choice: ToolChoiceOptions | ToolChoiceFunction | ToolChoiceMcp
      
      How the model chooses tools. Provide one of the string modes or force a specific function/MCP tool.
      - ToolChoiceOptions = :none | :auto | :required
        
        Controls which (if any) tool is called by the model.
        
        none means the model will not call any tool and instead generates a message.
        
        auto means the model can pick between generating a message or calling one or more tools.
        
        required means the model must call one or more tools.
        
        :none
        
        :auto
        
        :required
      - class ToolChoiceFunction
        
        Use this option to force the model to call a specific function.
        
        name: String
        
        The name of the function to call.
        
        type: :function
        
        For function calling, the type is always function.
        
        :function
      - class ToolChoiceMcp
        
        Use this option to force the model to call a specific tool on a remote MCP server.
        
        server_label: String
        
        The label of the MCP server to use.
        
        type: :mcp
        
        For MCP tools, the type is always mcp.
        
        :mcp
        
        name: String
        
        The name of the tool to call on the server.
    - tools: Array[RealtimeFunctionTool | RealtimeResponseCreateMcpTool]
      
      Tools available to the model.
      - class RealtimeFunctionTool
        
        description: String
        
        The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything).
        
        name: String
        
        The name of the function.
        
        parameters: untyped
        
        Parameters of the function in JSON Schema.
        
        type: :function
        
        The type of the tool, i.e. function.
        
        :function
      - class RealtimeResponseCreateMcpTool
        
        Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.
        
        server_label: String
        
        A label for this MCP server, used to identify it in tool calls.
        
        type: :mcp
        
        The type of the MCP tool. Always mcp.
        
        :mcp
        
        allowed_tools: Array[String] | McpToolFilter{ read_only, tool_names}
        
        List of allowed tool names or a filter object.
        
        McpAllowedTools = Array[String]
        
        A string array of allowed tool names
        
        class McpToolFilter
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        authorization: String
        
        An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.
        
        connector_id: :connector_dropbox | :connector_gmail | :connector_googlecalendar | 5 more
        
        Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here.
        
        Currently supported connector_id values are:
        
        Dropbox: connector_dropbox
        
        Gmail: connector_gmail
        
        Google Calendar: connector_googlecalendar
        
        Google Drive: connector_googledrive
        
        Microsoft Teams: connector_microsoftteams
        
        Outlook Calendar: connector_outlookcalendar
        
        Outlook Email: connector_outlookemail
        
        SharePoint: connector_sharepoint
        
        :connector_dropbox
        
        :connector_gmail
        
        :connector_googlecalendar
        
        :connector_googledrive
        
        :connector_microsoftteams
        
        :connector_outlookcalendar
        
        :connector_outlookemail
        
        :connector_sharepoint
        
        defer_loading: bool
        
        Whether this MCP tool is deferred and discovered via tool search.
        
        headers: Hash[Symbol, String]
        
        Optional HTTP headers to send to the MCP server. Use for authentication or other purposes.
        
        require_approval: McpToolApprovalFilter{ always, never} | :always | :never
        
        Specify which of the MCP server's tools require approval.
        
        class McpToolApprovalFilter
        
        Specify which of the MCP server's tools require approval. Can be always, never, or a filter object associated with tools that require approval.
        
        always: Always{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        never: Never{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        McpToolApprovalSetting = :always | :never
        
        Specify a single approval policy for all tools. One of always or never. When set to always, all tools will require approval. When set to never, all tools will not require approval.
        
        :always
        
        :never
        
        server_description: String
        
        Optional description of the MCP server, used to provide more context.
        
        server_url: String
        
        The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided.
        
        tunnel_id: String
        
        The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided.
- class SessionUpdateEvent
  
  Send this event to update the session’s configuration. The client may send this event at any time to update any field except for voice and model. voice can be updated only if there have been no other audio outputs yet.
  
  When the server receives a session.update, it will respond with a session.updated event showing the full, effective configuration. Only the fields that are present in the session.update are updated. To clear a field like instructions, pass an empty string. To clear a field like tools, pass an empty array. To clear a field like turn_detection, pass null.
  - session: RealtimeSessionCreateRequest | RealtimeTranscriptionSessionCreateRequest
    
    Update the Realtime session. Choose either a realtime session or a transcription session.
    - class RealtimeSessionCreateRequest
      
      Realtime session object configuration.
      - type: :realtime
        
        The type of session to create. Always realtime for the Realtime API.
        
        :realtime
      - audio: RealtimeAudioConfig
        
        Configuration for input and output audio.
        
        input: RealtimeAudioConfigInput
        
        format_: RealtimeAudioFormats
        
        The format of the input audio.
        
        noise_reduction: NoiseReduction{ type}
        
        Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
        
        transcription: AudioTranscription
        
        Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
        
        delay: :minimal | :low | :medium | 2 more
        
        Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
        
        :minimal
        
        :low
        
        :medium
        
        :high
        
        :xhigh
        
        language: String
        
        The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
        
        model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        String = String
        
        Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        :"whisper-1"
        
        :"gpt-4o-mini-transcribe"
        
        :"gpt-4o-mini-transcribe-2025-12-15"
        
        :"gpt-4o-transcribe"
        
        :"gpt-4o-transcribe-diarize"
        
        :"gpt-realtime-whisper"
        
        prompt: String
        
        An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
        
        turn_detection: RealtimeAudioInputTurnDetection
        
        Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
        
        Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
        
        Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
        
        For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
        
        class ServerVad
        
        Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
        
        type: :server_vad
        
        Type of turn detection, server_vad to turn on simple Server VAD.
        
        :server_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        idle_timeout_ms: Integer
        
        Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
        
        The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
        
        An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        prefix_padding_ms: Integer
        
        Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
        
        silence_duration_ms: Integer
        
        Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
        
        threshold: Float
        
        Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
        
        class SemanticVad
        
        Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
        
        type: :semantic_vad
        
        Type of turn detection, semantic_vad to turn on Semantic VAD.
        
        :semantic_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs.
        
        eagerness: :low | :medium | :high | :auto
        
        Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
        
        :low
        
        :medium
        
        :high
        
        :auto
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.
        
        output: RealtimeAudioConfigOutput
        
        format_: RealtimeAudioFormats
        
        The format of the output audio.
        
        speed: Float
        
        The speed of the model's spoken response as a multiple of the original speed. 1.0 is the default speed. 0.25 is the minimum speed. 1.5 is the maximum speed. This value can only be changed in between model turns, not while a response is in progress.
        
        This parameter is a post-processing adjustment to the audio after it is generated, it's also possible to prompt the model to speak faster or slower.
        
        voice: String | :alloy | :ash | :ballad | 7 more | ID{ id}
        
        The voice the model uses to respond. Supported built-in voices are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. You may also provide a custom voice object with an id, for example { "id": "voice_1234" }. Voice cannot be changed during the session once the model has responded with audio at least once. We recommend marin and cedar for best quality.
        
        String = String
        
        Voice = :alloy | :ash | :ballad | 7 more
        
        :alloy
        
        :ash
        
        :ballad
        
        :coral
        
        :echo
        
        :sage
        
        :shimmer
        
        :verse
        
        :marin
        
        :cedar
        
        class ID
        
        Custom voice reference.
        
        id: String
        
        The custom voice ID, e.g. voice_1234.
      - include: Array[:"item.input_audio_transcription.logprobs"]
        
        Additional fields to include in server outputs.
        
        item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
        
        :"item.input_audio_transcription.logprobs"
      - instructions: String
        
        The default system instructions (i.e. system message) prepended to model calls. This field allows the client to guide the model on desired responses. The model can be instructed on response content and format, (e.g. "be extremely succinct", "act friendly", "here are examples of good responses") and on audio behavior (e.g. "talk quickly", "inject emotion into your voice", "laugh frequently"). The instructions are not guaranteed to be followed by the model, but they provide guidance to the model on the desired behavior.
        
        Note that the server sets default instructions which will be used if this field is not set and are visible in the session.created event at the start of the session.
      - max_output_tokens: Integer | :inf
        
        Maximum number of output tokens for a single assistant response, inclusive of tool calls. Provide an integer between 1 and 4096 to limit output tokens, or inf for the maximum available tokens for a given model. Defaults to inf.
        
        Integer = Integer
        
        MaxOutputTokens = :inf
        
        :inf
      - model: String | :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
        
        The Realtime model used for this session.
        
        String = String
        
        Model = :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
        
        The Realtime model used for this session.
        
        :"gpt-realtime"
        
        :"gpt-realtime-1.5"
        
        :"gpt-realtime-2"
        
        :"gpt-realtime-2025-08-28"
        
        :"gpt-4o-realtime-preview"
        
        :"gpt-4o-realtime-preview-2024-10-01"
        
        :"gpt-4o-realtime-preview-2024-12-17"
        
        :"gpt-4o-realtime-preview-2025-06-03"
        
        :"gpt-4o-mini-realtime-preview"
        
        :"gpt-4o-mini-realtime-preview-2024-12-17"
        
        :"gpt-realtime-mini"
        
        :"gpt-realtime-mini-2025-10-06"
        
        :"gpt-realtime-mini-2025-12-15"
        
        :"gpt-audio-1.5"
        
        :"gpt-audio-mini"
        
        :"gpt-audio-mini-2025-10-06"
        
        :"gpt-audio-mini-2025-12-15"
      - output_modalities: Array[:text | :audio]
        
        The set of modalities the model can respond with. It defaults to ["audio"], indicating that the model will respond with audio plus a transcript. ["text"] can be used to make the model respond with text only. It is not possible to request both text and audio at the same time.
        
        :text
        
        :audio
      - parallel_tool_calls: bool
        
        Whether the model may call multiple tools in parallel. Only supported by reasoning Realtime models such as gpt-realtime-2.
      - prompt: ResponsePrompt
        
        Reference to a prompt template and its variables. Learn more.
      - reasoning: RealtimeReasoning
        
        Configuration for reasoning-capable Realtime models such as gpt-realtime-2.
      - tool_choice: RealtimeToolChoiceConfig
        
        How the model chooses tools. Provide one of the string modes or force a specific function/MCP tool.
        
        ToolChoiceOptions = :none | :auto | :required
        
        Controls which (if any) tool is called by the model.
        
        none means the model will not call any tool and instead generates a message.
        
        auto means the model can pick between generating a message or calling one or more tools.
        
        required means the model must call one or more tools.
        
        class ToolChoiceFunction
        
        Use this option to force the model to call a specific function.
        
        class ToolChoiceMcp
        
        Use this option to force the model to call a specific tool on a remote MCP server.
      - tools: RealtimeToolsConfig
        
        Tools available to the model.
        
        class RealtimeFunctionTool
        
        class Mcp
        
        Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.
        
        server_label: String
        
        A label for this MCP server, used to identify it in tool calls.
        
        type: :mcp
        
        The type of the MCP tool. Always mcp.
        
        :mcp
        
        allowed_tools: Array[String] | McpToolFilter{ read_only, tool_names}
        
        List of allowed tool names or a filter object.
        
        McpAllowedTools = Array[String]
        
        A string array of allowed tool names
        
        class McpToolFilter
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        authorization: String
        
        An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.
        
        connector_id: :connector_dropbox | :connector_gmail | :connector_googlecalendar | 5 more
        
        Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here.
        
        Currently supported connector_id values are:
        
        Dropbox: connector_dropbox
        
        Gmail: connector_gmail
        
        Google Calendar: connector_googlecalendar
        
        Google Drive: connector_googledrive
        
        Microsoft Teams: connector_microsoftteams
        
        Outlook Calendar: connector_outlookcalendar
        
        Outlook Email: connector_outlookemail
        
        SharePoint: connector_sharepoint
        
        :connector_dropbox
        
        :connector_gmail
        
        :connector_googlecalendar
        
        :connector_googledrive
        
        :connector_microsoftteams
        
        :connector_outlookcalendar
        
        :connector_outlookemail
        
        :connector_sharepoint
        
        defer_loading: bool
        
        Whether this MCP tool is deferred and discovered via tool search.
        
        headers: Hash[Symbol, String]
        
        Optional HTTP headers to send to the MCP server. Use for authentication or other purposes.
        
        require_approval: McpToolApprovalFilter{ always, never} | :always | :never
        
        Specify which of the MCP server's tools require approval.
        
        class McpToolApprovalFilter
        
        Specify which of the MCP server's tools require approval. Can be always, never, or a filter object associated with tools that require approval.
        
        always: Always{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        never: Never{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        McpToolApprovalSetting = :always | :never
        
        Specify a single approval policy for all tools. One of always or never. When set to always, all tools will require approval. When set to never, all tools will not require approval.
        
        :always
        
        :never
        
        server_description: String
        
        Optional description of the MCP server, used to provide more context.
        
        server_url: String
        
        The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided.
        
        tunnel_id: String
        
        The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided.
      - tracing: RealtimeTracingConfig
        
        Realtime API can write session traces to the Traces Dashboard. Set to null to disable tracing. Once tracing is enabled for a session, the configuration cannot be modified.
        
        auto will create a trace for the session with default values for the workflow name, group id, and metadata.
        
        RealtimeTracingConfig = :auto
        
        Enables tracing and sets default values for tracing configuration options. Always auto.
        
        :auto
        
        class TracingConfiguration
        
        Granular configuration for tracing.
        
        group_id: String
        
        The group id to attach to this trace to enable filtering and grouping in the Traces Dashboard.
        
        metadata: untyped
        
        The arbitrary metadata to attach to this trace to enable filtering in the Traces Dashboard.
        
        workflow_name: String
        
        The name of the workflow to attach to this trace. This is used to name the trace in the Traces Dashboard.
      - truncation: RealtimeTruncation
        
        When the number of tokens in a conversation exceeds the model's input token limit, the conversation be truncated, meaning messages (starting from the oldest) will not be included in the model's context. A 32k context model with 4,096 max output tokens can only include 28,224 tokens in the context before truncation occurs.
        
        Clients can configure truncation behavior to truncate with a lower max token limit, which is an effective way to control token usage and cost.
        
        Truncation will reduce the number of cached tokens on the next turn (busting the cache), since messages are dropped from the beginning of the context. However, clients can also configure truncation to retain messages up to a fraction of the maximum context size, which will reduce the need for future truncations and thus improve the cache rate.
        
        Truncation can be disabled entirely, which means the server will never truncate but would instead return an error if the conversation exceeds the model's input token limit.
        
        RealtimeTruncationStrategy = :auto | :disabled
        
        The truncation strategy to use for the session. auto is the default truncation strategy. disabled will disable truncation and emit errors when the conversation exceeds the input token limit.
        
        :auto
        
        :disabled
        
        class RealtimeTruncationRetentionRatio
        
        Retain a fraction of the conversation tokens when the conversation exceeds the input token limit. This allows you to amortize truncations across multiple turns, which can help improve cached token usage.
        
        retention_ratio: Float
        
        Fraction of post-instruction conversation tokens to retain (0.0 - 1.0) when the conversation exceeds the input token limit. Setting this to 0.8 means that messages will be dropped until 80% of the maximum allowed tokens are used. This helps reduce the frequency of truncations and improve cache rates.
        
        type: :retention_ratio
        
        Use retention ratio truncation.
        
        :retention_ratio
        
        token_limits: TokenLimits{ post_instructions}
        
        Optional custom token limits for this truncation strategy. If not provided, the model's default token limits will be used.
        
        post_instructions: Integer
        
        Maximum tokens allowed in the conversation after instructions (which including tool definitions). For example, setting this to 5,000 would mean that truncation would occur when the conversation exceeds 5,000 tokens after instructions. This cannot be higher than the model's context window size minus the maximum output tokens.
    - class RealtimeTranscriptionSessionCreateRequest
      
      Realtime transcription session object configuration.
      - type: :transcription
        
        The type of session to create. Always transcription for transcription sessions.
        
        :transcription
      - audio: RealtimeTranscriptionSessionAudio
        
        Configuration for input and output audio.
        
        input: RealtimeTranscriptionSessionAudioInput
        
        format_: RealtimeAudioFormats
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        noise_reduction: NoiseReduction{ type}
        
        Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        transcription: AudioTranscription
        
        Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
        
        turn_detection: RealtimeTranscriptionSessionAudioInputTurnDetection
        
        Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
        
        Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
        
        Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
        
        For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
        
        class ServerVad
        
        Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
        
        type: :server_vad
        
        Type of turn detection, server_vad to turn on simple Server VAD.
        
        :server_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        idle_timeout_ms: Integer
        
        Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
        
        The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
        
        An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        prefix_padding_ms: Integer
        
        Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
        
        silence_duration_ms: Integer
        
        Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
        
        threshold: Float
        
        Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
        
        class SemanticVad
        
        Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
        
        type: :semantic_vad
        
        Type of turn detection, semantic_vad to turn on Semantic VAD.
        
        :semantic_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs.
        
        eagerness: :low | :medium | :high | :auto
        
        Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
        
        :low
        
        :medium
        
        :high
        
        :auto
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.
      - include: Array[:"item.input_audio_transcription.logprobs"]
        
        Additional fields to include in server outputs.
        
        item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
        
        :"item.input_audio_transcription.logprobs"
  - type: :"session.update"
    
    The event type, must be session.update.
    - :"session.update"
  - event_id: String
    
    Optional client-generated ID used to identify this event. This is an arbitrary string that a client may assign. It will be passed back if there is an error with the event, but the corresponding session.updated event will not include it.

Realtime Conversation Item Assistant Message

class RealtimeConversationItemAssistantMessage

An assistant message item in a Realtime conversation.
- content: Array[Content{ audio, text, transcript, type}]
  
  The content of the message.
  - audio: String
    
    Base64-encoded audio bytes, these will be parsed as the format specified in the session output audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
  - text: String
    
    The text content.
  - transcript: String
    
    The transcript of the audio content, this will always be present if the output type is audio.
  - type: :output_text | :output_audio
    
    The content type, output_text or output_audio depending on the session output_modalities configuration.
    - :output_text
    - :output_audio
- role: :assistant
  
  The role of the message sender. Always assistant.
  - :assistant
- type: :message
  
  The type of the item. Always message.
  - :message
- id: String
  
  The unique ID of the item. This may be provided by the client or generated by the server.
- object: :"realtime.item"
  
  Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
  - :"realtime.item"
- status: :completed | :incomplete | :in_progress
  
  The status of the item. Has no effect on the conversation.
  - :completed
  - :incomplete
  - :in_progress

Realtime Conversation Item Function Call

class RealtimeConversationItemFunctionCall

A function call item in a Realtime conversation.
- arguments: String
  
  The arguments of the function call. This is a JSON-encoded string representing the arguments passed to the function, for example {"arg1": "value1", "arg2": 42}.
- name: String
  
  The name of the function being called.
- type: :function_call
  
  The type of the item. Always function_call.
  - :function_call
- id: String
  
  The unique ID of the item. This may be provided by the client or generated by the server.
- call_id: String
  
  The ID of the function call.
- object: :"realtime.item"
  
  Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
  - :"realtime.item"
- status: :completed | :incomplete | :in_progress
  
  The status of the item. Has no effect on the conversation.
  - :completed
  - :incomplete
  - :in_progress

Realtime Conversation Item Function Call Output

class RealtimeConversationItemFunctionCallOutput

A function call output item in a Realtime conversation.
- call_id: String
  
  The ID of the function call this output is for.
- output: String
  
  The output of the function call, this is free text and can contain any information or simply be empty.
- type: :function_call_output
  
  The type of the item. Always function_call_output.
  - :function_call_output
- id: String
  
  The unique ID of the item. This may be provided by the client or generated by the server.
- object: :"realtime.item"
  
  Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
  - :"realtime.item"
- status: :completed | :incomplete | :in_progress
  
  The status of the item. Has no effect on the conversation.
  - :completed
  - :incomplete
  - :in_progress

Realtime Conversation Item System Message

class RealtimeConversationItemSystemMessage

A system message in a Realtime conversation can be used to provide additional context or instructions to the model. This is similar but distinct from the instruction prompt provided at the start of a conversation, as system messages can be added at any point in the conversation. For major changes to the conversation's behavior, use instructions, but for smaller updates (e.g. "the user is now asking about a different topic"), use system messages.
- content: Array[Content{ text, type}]
  
  The content of the message.
  - text: String
    
    The text content.
  - type: :input_text
    
    The content type. Always input_text for system messages.
    - :input_text
- role: :system
  
  The role of the message sender. Always system.
  - :system
- type: :message
  
  The type of the item. Always message.
  - :message
- id: String
  
  The unique ID of the item. This may be provided by the client or generated by the server.
- object: :"realtime.item"
  
  Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
  - :"realtime.item"
- status: :completed | :incomplete | :in_progress
  
  The status of the item. Has no effect on the conversation.
  - :completed
  - :incomplete
  - :in_progress

Realtime Conversation Item User Message

class RealtimeConversationItemUserMessage

A user message item in a Realtime conversation.
- content: Array[Content{ audio, detail, image_url, 3 more}]
  
  The content of the message.
  - audio: String
    
    Base64-encoded audio bytes (for input_audio), these will be parsed as the format specified in the session input audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
  - detail: :auto | :low | :high
    
    The detail level of the image (for input_image). auto will default to high.
    - :auto
    - :low
    - :high
  - image_url: String
    
    Base64-encoded image bytes (for input_image) as a data URI. For example data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA.... Supported formats are PNG and JPEG.
  - text: String
    
    The text content (for input_text).
  - transcript: String
    
    Transcript of the audio (for input_audio). This is not sent to the model, but will be attached to the message item for reference.
  - type: :input_text | :input_audio | :input_image
    
    The content type (input_text, input_audio, or input_image).
    - :input_text
    - :input_audio
    - :input_image
- role: :user
  
  The role of the message sender. Always user.
  - :user
- type: :message
  
  The type of the item. Always message.
  - :message
- id: String
  
  The unique ID of the item. This may be provided by the client or generated by the server.
- object: :"realtime.item"
  
  Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
  - :"realtime.item"
- status: :completed | :incomplete | :in_progress
  
  The status of the item. Has no effect on the conversation.
  - :completed
  - :incomplete
  - :in_progress

Realtime Error

class RealtimeError

Details of the error.
- message: String
  
  A human-readable error message.
- type: String
  
  The type of error (e.g., "invalid_request_error", "server_error").
- code: String
  
  Error code, if any.
- event_id: String
  
  The event_id of the client event that caused the error, if applicable.
- param: String
  
  Parameter related to the error, if any.

Realtime Error Event

class RealtimeErrorEvent

Returned when an error occurs, which could be a client problem or a server problem. Most errors are recoverable and the session will stay open, we recommend to implementors to monitor and log error messages by default.
- error: RealtimeError
  
  Details of the error.
  - message: String
    
    A human-readable error message.
  - type: String
    
    The type of error (e.g., "invalid_request_error", "server_error").
  - code: String
    
    Error code, if any.
  - event_id: String
    
    The event_id of the client event that caused the error, if applicable.
  - param: String
    
    Parameter related to the error, if any.
- event_id: String
  
  The unique ID of the server event.
- type: :error
  
  The event type, must be error.
  - :error

Realtime Function Tool

class RealtimeFunctionTool
- description: String
  
  The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything).
- name: String
  
  The name of the function.
- parameters: untyped
  
  Parameters of the function in JSON Schema.
- type: :function
  
  The type of the tool, i.e. function.
  - :function

Realtime Mcp Approval Request

class RealtimeMcpApprovalRequest

A Realtime item requesting human approval of a tool invocation.
- id: String
  
  The unique ID of the approval request.
- arguments: String
  
  A JSON string of arguments for the tool.
- name: String
  
  The name of the tool to run.
- server_label: String
  
  The label of the MCP server making the request.
- type: :mcp_approval_request
  
  The type of the item. Always mcp_approval_request.
  - :mcp_approval_request

Realtime Mcp Approval Response

class RealtimeMcpApprovalResponse

A Realtime item responding to an MCP approval request.
- id: String
  
  The unique ID of the approval response.
- approval_request_id: String
  
  The ID of the approval request being answered.
- approve: bool
  
  Whether the request was approved.
- type: :mcp_approval_response
  
  The type of the item. Always mcp_approval_response.
  - :mcp_approval_response
- reason: String
  
  Optional reason for the decision.

Realtime Mcp List Tools

class RealtimeMcpListTools

A Realtime item listing tools available on an MCP server.
- server_label: String
  
  The label of the MCP server.
- tools: Array[Tool{ input_schema, name, annotations, description}]
  
  The tools available on the server.
  - input_schema: untyped
    
    The JSON schema describing the tool's input.
  - name: String
    
    The name of the tool.
  - annotations: untyped
    
    Additional annotations about the tool.
  - description: String
    
    The description of the tool.
- type: :mcp_list_tools
  
  The type of the item. Always mcp_list_tools.
  - :mcp_list_tools
- id: String
  
  The unique ID of the list.

Realtime Mcp Protocol Error

class RealtimeMcpProtocolError
- code: Integer
- message: String
- type: :protocol_error
  - :protocol_error

Realtime Mcp Tool Call

class RealtimeMcpToolCall

A Realtime item representing an invocation of a tool on an MCP server.
- id: String
  
  The unique ID of the tool call.
- arguments: String
  
  A JSON string of the arguments passed to the tool.
- name: String
  
  The name of the tool that was run.
- server_label: String
  
  The label of the MCP server running the tool.
- type: :mcp_call
  
  The type of the item. Always mcp_call.
  - :mcp_call
- approval_request_id: String
  
  The ID of an associated approval request, if any.
- error: RealtimeMcpProtocolError | RealtimeMcpToolExecutionError | RealtimeMcphttpError
  
  The error from the tool call, if any.
  - class RealtimeMcpProtocolError
    - code: Integer
    - message: String
    - type: :protocol_error
      - :protocol_error
  - class RealtimeMcpToolExecutionError
    - message: String
    - type: :tool_execution_error
      - :tool_execution_error
  - class RealtimeMcphttpError
    - code: Integer
    - message: String
    - type: :http_error
      - :http_error
- output: String
  
  The output from the tool call.

Realtime Mcp Tool Execution Error

class RealtimeMcpToolExecutionError
- message: String
- type: :tool_execution_error
  - :tool_execution_error

Realtime Mcphttp Error

class RealtimeMcphttpError
- code: Integer
- message: String
- type: :http_error
  - :http_error

Realtime Reasoning

class RealtimeReasoning

Configuration for reasoning-capable Realtime models such as gpt-realtime-2.
- effort: RealtimeReasoningEffort
  
  Constrains effort on reasoning for reasoning-capable Realtime models such as gpt-realtime-2.
  - :minimal
  - :low
  - :medium
  - :high
  - :xhigh

Realtime Reasoning Effort

RealtimeReasoningEffort = :minimal | :low | :medium | 2 more

Constrains effort on reasoning for reasoning-capable Realtime models such as gpt-realtime-2.
- :minimal
- :low
- :medium
- :high
- :xhigh

Realtime Response

class RealtimeResponse

The response resource.
- id: String
  
  The unique ID of the response, will look like resp_1234.
- audio: Audio{ output}
  
  Configuration for audio output.
  - output: Output{ format_, voice}
    - format_: RealtimeAudioFormats
      
      The format of the output audio.
      - class AudioPCM
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
        
        type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
      - class AudioPCMU
        
        The G.711 μ-law format.
        
        type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
      - class AudioPCMA
        
        The G.711 A-law format.
        
        type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
    - voice: String | :alloy | :ash | :ballad | 7 more
      
      The voice the model uses to respond. Voice cannot be changed during the session once the model has responded with audio at least once. Current voice options are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. We recommend marin and cedar for best quality.
      - String = String
      - Voice = :alloy | :ash | :ballad | 7 more
        
        The voice the model uses to respond. Voice cannot be changed during the session once the model has responded with audio at least once. Current voice options are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. We recommend marin and cedar for best quality.
        
        :alloy
        
        :ash
        
        :ballad
        
        :coral
        
        :echo
        
        :sage
        
        :shimmer
        
        :verse
        
        :marin
        
        :cedar
- conversation_id: String
  
  Which conversation the response is added to, determined by the conversation field in the response.create event. If auto, the response will be added to the default conversation and the value of conversation_id will be an id like conv_1234. If none, the response will not be added to any conversation and the value of conversation_id will be null. If responses are being triggered automatically by VAD the response will be added to the default conversation
- max_output_tokens: Integer | :inf
  
  Maximum number of output tokens for a single assistant response, inclusive of tool calls, that was used in this response.
  - Integer = Integer
  - MaxOutputTokens = :inf
    - :inf
- metadata: Metadata
  
  Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard.
  
  Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.
- object: :"realtime.response"
  
  The object type, must be realtime.response.
  - :"realtime.response"
- output: Array[ConversationItem]
  
  The list of output items generated by the response.
  - class RealtimeConversationItemSystemMessage
    
    A system message in a Realtime conversation can be used to provide additional context or instructions to the model. This is similar but distinct from the instruction prompt provided at the start of a conversation, as system messages can be added at any point in the conversation. For major changes to the conversation's behavior, use instructions, but for smaller updates (e.g. "the user is now asking about a different topic"), use system messages.
    - content: Array[Content{ text, type}]
      
      The content of the message.
      - text: String
        
        The text content.
      - type: :input_text
        
        The content type. Always input_text for system messages.
        
        :input_text
    - role: :system
      
      The role of the message sender. Always system.
      - :system
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemUserMessage
    
    A user message item in a Realtime conversation.
    - content: Array[Content{ audio, detail, image_url, 3 more}]
      
      The content of the message.
      - audio: String
        
        Base64-encoded audio bytes (for input_audio), these will be parsed as the format specified in the session input audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
      - detail: :auto | :low | :high
        
        The detail level of the image (for input_image). auto will default to high.
        
        :auto
        
        :low
        
        :high
      - image_url: String
        
        Base64-encoded image bytes (for input_image) as a data URI. For example data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA.... Supported formats are PNG and JPEG.
      - text: String
        
        The text content (for input_text).
      - transcript: String
        
        Transcript of the audio (for input_audio). This is not sent to the model, but will be attached to the message item for reference.
      - type: :input_text | :input_audio | :input_image
        
        The content type (input_text, input_audio, or input_image).
        
        :input_text
        
        :input_audio
        
        :input_image
    - role: :user
      
      The role of the message sender. Always user.
      - :user
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemAssistantMessage
    
    An assistant message item in a Realtime conversation.
    - content: Array[Content{ audio, text, transcript, type}]
      
      The content of the message.
      - audio: String
        
        Base64-encoded audio bytes, these will be parsed as the format specified in the session output audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
      - text: String
        
        The text content.
      - transcript: String
        
        The transcript of the audio content, this will always be present if the output type is audio.
      - type: :output_text | :output_audio
        
        The content type, output_text or output_audio depending on the session output_modalities configuration.
        
        :output_text
        
        :output_audio
    - role: :assistant
      
      The role of the message sender. Always assistant.
      - :assistant
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemFunctionCall
    
    A function call item in a Realtime conversation.
    - arguments: String
      
      The arguments of the function call. This is a JSON-encoded string representing the arguments passed to the function, for example {"arg1": "value1", "arg2": 42}.
    - name: String
      
      The name of the function being called.
    - type: :function_call
      
      The type of the item. Always function_call.
      - :function_call
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - call_id: String
      
      The ID of the function call.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemFunctionCallOutput
    
    A function call output item in a Realtime conversation.
    - call_id: String
      
      The ID of the function call this output is for.
    - output: String
      
      The output of the function call, this is free text and can contain any information or simply be empty.
    - type: :function_call_output
      
      The type of the item. Always function_call_output.
      - :function_call_output
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeMcpApprovalResponse
    
    A Realtime item responding to an MCP approval request.
    - id: String
      
      The unique ID of the approval response.
    - approval_request_id: String
      
      The ID of the approval request being answered.
    - approve: bool
      
      Whether the request was approved.
    - type: :mcp_approval_response
      
      The type of the item. Always mcp_approval_response.
      - :mcp_approval_response
    - reason: String
      
      Optional reason for the decision.
  - class RealtimeMcpListTools
    
    A Realtime item listing tools available on an MCP server.
    - server_label: String
      
      The label of the MCP server.
    - tools: Array[Tool{ input_schema, name, annotations, description}]
      
      The tools available on the server.
      - input_schema: untyped
        
        The JSON schema describing the tool's input.
      - name: String
        
        The name of the tool.
      - annotations: untyped
        
        Additional annotations about the tool.
      - description: String
        
        The description of the tool.
    - type: :mcp_list_tools
      
      The type of the item. Always mcp_list_tools.
      - :mcp_list_tools
    - id: String
      
      The unique ID of the list.
  - class RealtimeMcpToolCall
    
    A Realtime item representing an invocation of a tool on an MCP server.
    - id: String
      
      The unique ID of the tool call.
    - arguments: String
      
      A JSON string of the arguments passed to the tool.
    - name: String
      
      The name of the tool that was run.
    - server_label: String
      
      The label of the MCP server running the tool.
    - type: :mcp_call
      
      The type of the item. Always mcp_call.
      - :mcp_call
    - approval_request_id: String
      
      The ID of an associated approval request, if any.
    - error: RealtimeMcpProtocolError | RealtimeMcpToolExecutionError | RealtimeMcphttpError
      
      The error from the tool call, if any.
      - class RealtimeMcpProtocolError
        
        code: Integer
        
        message: String
        
        type: :protocol_error
        
        :protocol_error
      - class RealtimeMcpToolExecutionError
        
        message: String
        
        type: :tool_execution_error
        
        :tool_execution_error
      - class RealtimeMcphttpError
        
        code: Integer
        
        message: String
        
        type: :http_error
        
        :http_error
    - output: String
      
      The output from the tool call.
  - class RealtimeMcpApprovalRequest
    
    A Realtime item requesting human approval of a tool invocation.
    - id: String
      
      The unique ID of the approval request.
    - arguments: String
      
      A JSON string of arguments for the tool.
    - name: String
      
      The name of the tool to run.
    - server_label: String
      
      The label of the MCP server making the request.
    - type: :mcp_approval_request
      
      The type of the item. Always mcp_approval_request.
      - :mcp_approval_request
- output_modalities: Array[:text | :audio]
  
  The set of modalities the model used to respond, currently the only possible values are [\"audio\"], [\"text\"]. Audio output always include a text transcript. Setting the output to mode text will disable audio output from the model.
  - :text
  - :audio
- status: :completed | :cancelled | :failed | 2 more
  
  The final status of the response (completed, cancelled, failed, or incomplete, in_progress).
  - :completed
  - :cancelled
  - :failed
  - :incomplete
  - :in_progress
- status_details: RealtimeResponseStatus
  
  Additional details about the status.
  - error: Error{ code, type}
    
    A description of the error that caused the response to fail, populated when the status is failed.
    - code: String
      
      Error code, if any.
    - type: String
      
      The type of error.
  - reason: :turn_detected | :client_cancelled | :max_output_tokens | :content_filter
    
    The reason the Response did not complete. For a cancelled Response, one of turn_detected (the server VAD detected a new start of speech) or client_cancelled (the client sent a cancel event). For an incomplete Response, one of max_output_tokens or content_filter (the server-side safety filter activated and cut off the response).
    - :turn_detected
    - :client_cancelled
    - :max_output_tokens
    - :content_filter
  - type: :completed | :cancelled | :incomplete | :failed
    
    The type of error that caused the response to fail, corresponding with the status field (completed, cancelled, incomplete, failed).
    - :completed
    - :cancelled
    - :incomplete
    - :failed
- usage: RealtimeResponseUsage
  
  Usage statistics for the Response, this will correspond to billing. A Realtime API session will maintain a conversation context and append new Items to the Conversation, thus output from previous turns (text and audio tokens) will become the input for later turns.
  - input_token_details: RealtimeResponseUsageInputTokenDetails
    
    Details about the input tokens used in the Response. Cached tokens are tokens from previous turns in the conversation that are included as context for the current response. Cached tokens here are counted as a subset of input tokens, meaning input tokens will include cached and uncached tokens.
    - audio_tokens: Integer
      
      The number of audio tokens used as input for the Response.
    - cached_tokens: Integer
      
      The number of cached tokens used as input for the Response.
    - cached_tokens_details: CachedTokensDetails{ audio_tokens, image_tokens, text_tokens}
      
      Details about the cached tokens used as input for the Response.
      - audio_tokens: Integer
        
        The number of cached audio tokens used as input for the Response.
      - image_tokens: Integer
        
        The number of cached image tokens used as input for the Response.
      - text_tokens: Integer
        
        The number of cached text tokens used as input for the Response.
    - image_tokens: Integer
      
      The number of image tokens used as input for the Response.
    - text_tokens: Integer
      
      The number of text tokens used as input for the Response.
  - input_tokens: Integer
    
    The number of input tokens used in the Response, including text and audio tokens.
  - output_token_details: RealtimeResponseUsageOutputTokenDetails
    
    Details about the output tokens used in the Response.
    - audio_tokens: Integer
      
      The number of audio tokens used in the Response.
    - text_tokens: Integer
      
      The number of text tokens used in the Response.
  - output_tokens: Integer
    
    The number of output tokens sent in the Response, including text and audio tokens.
  - total_tokens: Integer
    
    The total number of tokens in the Response including input and output text and audio tokens.

Realtime Response Create Audio Output

class RealtimeResponseCreateAudioOutput

Configuration for audio input and output.
- output: Output{ format_, voice}
  - format_: RealtimeAudioFormats
    
    The format of the output audio.
    - class AudioPCM
      
      The PCM audio format. Only a 24kHz sample rate is supported.
      - rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
      - type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
    - class AudioPCMU
      
      The G.711 μ-law format.
      - type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
    - class AudioPCMA
      
      The G.711 A-law format.
      - type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
  - voice: String | :alloy | :ash | :ballad | 7 more | ID{ id}
    
    The voice the model uses to respond. Supported built-in voices are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. You may also provide a custom voice object with an id, for example { "id": "voice_1234" }. Voice cannot be changed during the session once the model has responded with audio at least once. We recommend marin and cedar for best quality.
    - String = String
    - Voice = :alloy | :ash | :ballad | 7 more
      - :alloy
      - :ash
      - :ballad
      - :coral
      - :echo
      - :sage
      - :shimmer
      - :verse
      - :marin
      - :cedar
    - class ID
      
      Custom voice reference.
      - id: String
        
        The custom voice ID, e.g. voice_1234.

Realtime Response Create Mcp Tool

class RealtimeResponseCreateMcpTool

Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.
- server_label: String
  
  A label for this MCP server, used to identify it in tool calls.
- type: :mcp
  
  The type of the MCP tool. Always mcp.
  - :mcp
- allowed_tools: Array[String] | McpToolFilter{ read_only, tool_names}
  
  List of allowed tool names or a filter object.
  - McpAllowedTools = Array[String]
    
    A string array of allowed tool names
  - class McpToolFilter
    
    A filter object to specify which tools are allowed.
    - read_only: bool
      
      Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
    - tool_names: Array[String]
      
      List of allowed tool names.
- authorization: String
  
  An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.
- connector_id: :connector_dropbox | :connector_gmail | :connector_googlecalendar | 5 more
  
  Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here.
  
  Currently supported connector_id values are:
  - Dropbox: connector_dropbox
  - Gmail: connector_gmail
  - Google Calendar: connector_googlecalendar
  - Google Drive: connector_googledrive
  - Microsoft Teams: connector_microsoftteams
  - Outlook Calendar: connector_outlookcalendar
  - Outlook Email: connector_outlookemail
  - SharePoint: connector_sharepoint
  - :connector_dropbox
  - :connector_gmail
  - :connector_googlecalendar
  - :connector_googledrive
  - :connector_microsoftteams
  - :connector_outlookcalendar
  - :connector_outlookemail
  - :connector_sharepoint
- defer_loading: bool
  
  Whether this MCP tool is deferred and discovered via tool search.
- headers: Hash[Symbol, String]
  
  Optional HTTP headers to send to the MCP server. Use for authentication or other purposes.
- require_approval: McpToolApprovalFilter{ always, never} | :always | :never
  
  Specify which of the MCP server's tools require approval.
  - class McpToolApprovalFilter
    
    Specify which of the MCP server's tools require approval. Can be always, never, or a filter object associated with tools that require approval.
    - always: Always{ read_only, tool_names}
      
      A filter object to specify which tools are allowed.
      - read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
      - tool_names: Array[String]
        
        List of allowed tool names.
    - never: Never{ read_only, tool_names}
      
      A filter object to specify which tools are allowed.
      - read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
      - tool_names: Array[String]
        
        List of allowed tool names.
  - McpToolApprovalSetting = :always | :never
    
    Specify a single approval policy for all tools. One of always or never. When set to always, all tools will require approval. When set to never, all tools will not require approval.
    - :always
    - :never
- server_description: String
  
  Optional description of the MCP server, used to provide more context.
- server_url: String
  
  The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided.
- tunnel_id: String
  
  The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided.

Realtime Response Create Params

class RealtimeResponseCreateParams

Create a new Realtime response with these parameters
- audio: RealtimeResponseCreateAudioOutput
  
  Configuration for audio input and output.
  - output: Output{ format_, voice}
    - format_: RealtimeAudioFormats
      
      The format of the output audio.
      - class AudioPCM
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
        
        type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
      - class AudioPCMU
        
        The G.711 μ-law format.
        
        type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
      - class AudioPCMA
        
        The G.711 A-law format.
        
        type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
    - voice: String | :alloy | :ash | :ballad | 7 more | ID{ id}
      
      The voice the model uses to respond. Supported built-in voices are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. You may also provide a custom voice object with an id, for example { "id": "voice_1234" }. Voice cannot be changed during the session once the model has responded with audio at least once. We recommend marin and cedar for best quality.
      - String = String
      - Voice = :alloy | :ash | :ballad | 7 more
        
        :alloy
        
        :ash
        
        :ballad
        
        :coral
        
        :echo
        
        :sage
        
        :shimmer
        
        :verse
        
        :marin
        
        :cedar
      - class ID
        
        Custom voice reference.
        
        id: String
        
        The custom voice ID, e.g. voice_1234.
- conversation: String | :auto | :none
  
  Controls which conversation the response is added to. Currently supports auto and none, with auto as the default value. The auto value means that the contents of the response will be added to the default conversation. Set this to none to create an out-of-band response which will not add items to default conversation.
  - String = String
  - Conversation = :auto | :none
    
    Controls which conversation the response is added to. Currently supports auto and none, with auto as the default value. The auto value means that the contents of the response will be added to the default conversation. Set this to none to create an out-of-band response which will not add items to default conversation.
    - :auto
    - :none
- input: Array[ConversationItem]
  
  Input items to include in the prompt for the model. Using this field creates a new context for this Response instead of using the default conversation. An empty array [] will clear the context for this Response. Note that this can include references to items that previously appeared in the session using their id.
  - class RealtimeConversationItemSystemMessage
    
    A system message in a Realtime conversation can be used to provide additional context or instructions to the model. This is similar but distinct from the instruction prompt provided at the start of a conversation, as system messages can be added at any point in the conversation. For major changes to the conversation's behavior, use instructions, but for smaller updates (e.g. "the user is now asking about a different topic"), use system messages.
    - content: Array[Content{ text, type}]
      
      The content of the message.
      - text: String
        
        The text content.
      - type: :input_text
        
        The content type. Always input_text for system messages.
        
        :input_text
    - role: :system
      
      The role of the message sender. Always system.
      - :system
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemUserMessage
    
    A user message item in a Realtime conversation.
    - content: Array[Content{ audio, detail, image_url, 3 more}]
      
      The content of the message.
      - audio: String
        
        Base64-encoded audio bytes (for input_audio), these will be parsed as the format specified in the session input audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
      - detail: :auto | :low | :high
        
        The detail level of the image (for input_image). auto will default to high.
        
        :auto
        
        :low
        
        :high
      - image_url: String
        
        Base64-encoded image bytes (for input_image) as a data URI. For example data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA.... Supported formats are PNG and JPEG.
      - text: String
        
        The text content (for input_text).
      - transcript: String
        
        Transcript of the audio (for input_audio). This is not sent to the model, but will be attached to the message item for reference.
      - type: :input_text | :input_audio | :input_image
        
        The content type (input_text, input_audio, or input_image).
        
        :input_text
        
        :input_audio
        
        :input_image
    - role: :user
      
      The role of the message sender. Always user.
      - :user
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemAssistantMessage
    
    An assistant message item in a Realtime conversation.
    - content: Array[Content{ audio, text, transcript, type}]
      
      The content of the message.
      - audio: String
        
        Base64-encoded audio bytes, these will be parsed as the format specified in the session output audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
      - text: String
        
        The text content.
      - transcript: String
        
        The transcript of the audio content, this will always be present if the output type is audio.
      - type: :output_text | :output_audio
        
        The content type, output_text or output_audio depending on the session output_modalities configuration.
        
        :output_text
        
        :output_audio
    - role: :assistant
      
      The role of the message sender. Always assistant.
      - :assistant
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemFunctionCall
    
    A function call item in a Realtime conversation.
    - arguments: String
      
      The arguments of the function call. This is a JSON-encoded string representing the arguments passed to the function, for example {"arg1": "value1", "arg2": 42}.
    - name: String
      
      The name of the function being called.
    - type: :function_call
      
      The type of the item. Always function_call.
      - :function_call
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - call_id: String
      
      The ID of the function call.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemFunctionCallOutput
    
    A function call output item in a Realtime conversation.
    - call_id: String
      
      The ID of the function call this output is for.
    - output: String
      
      The output of the function call, this is free text and can contain any information or simply be empty.
    - type: :function_call_output
      
      The type of the item. Always function_call_output.
      - :function_call_output
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeMcpApprovalResponse
    
    A Realtime item responding to an MCP approval request.
    - id: String
      
      The unique ID of the approval response.
    - approval_request_id: String
      
      The ID of the approval request being answered.
    - approve: bool
      
      Whether the request was approved.
    - type: :mcp_approval_response
      
      The type of the item. Always mcp_approval_response.
      - :mcp_approval_response
    - reason: String
      
      Optional reason for the decision.
  - class RealtimeMcpListTools
    
    A Realtime item listing tools available on an MCP server.
    - server_label: String
      
      The label of the MCP server.
    - tools: Array[Tool{ input_schema, name, annotations, description}]
      
      The tools available on the server.
      - input_schema: untyped
        
        The JSON schema describing the tool's input.
      - name: String
        
        The name of the tool.
      - annotations: untyped
        
        Additional annotations about the tool.
      - description: String
        
        The description of the tool.
    - type: :mcp_list_tools
      
      The type of the item. Always mcp_list_tools.
      - :mcp_list_tools
    - id: String
      
      The unique ID of the list.
  - class RealtimeMcpToolCall
    
    A Realtime item representing an invocation of a tool on an MCP server.
    - id: String
      
      The unique ID of the tool call.
    - arguments: String
      
      A JSON string of the arguments passed to the tool.
    - name: String
      
      The name of the tool that was run.
    - server_label: String
      
      The label of the MCP server running the tool.
    - type: :mcp_call
      
      The type of the item. Always mcp_call.
      - :mcp_call
    - approval_request_id: String
      
      The ID of an associated approval request, if any.
    - error: RealtimeMcpProtocolError | RealtimeMcpToolExecutionError | RealtimeMcphttpError
      
      The error from the tool call, if any.
      - class RealtimeMcpProtocolError
        
        code: Integer
        
        message: String
        
        type: :protocol_error
        
        :protocol_error
      - class RealtimeMcpToolExecutionError
        
        message: String
        
        type: :tool_execution_error
        
        :tool_execution_error
      - class RealtimeMcphttpError
        
        code: Integer
        
        message: String
        
        type: :http_error
        
        :http_error
    - output: String
      
      The output from the tool call.
  - class RealtimeMcpApprovalRequest
    
    A Realtime item requesting human approval of a tool invocation.
    - id: String
      
      The unique ID of the approval request.
    - arguments: String
      
      A JSON string of arguments for the tool.
    - name: String
      
      The name of the tool to run.
    - server_label: String
      
      The label of the MCP server making the request.
    - type: :mcp_approval_request
      
      The type of the item. Always mcp_approval_request.
      - :mcp_approval_request
- instructions: String
  
  The default system instructions (i.e. system message) prepended to model calls. This field allows the client to guide the model on desired responses. The model can be instructed on response content and format, (e.g. "be extremely succinct", "act friendly", "here are examples of good responses") and on audio behavior (e.g. "talk quickly", "inject emotion into your voice", "laugh frequently"). The instructions are not guaranteed to be followed by the model, but they provide guidance to the model on the desired behavior. Note that the server sets default instructions which will be used if this field is not set and are visible in the session.created event at the start of the session.
- max_output_tokens: Integer | :inf
  
  Maximum number of output tokens for a single assistant response, inclusive of tool calls. Provide an integer between 1 and 4096 to limit output tokens, or inf for the maximum available tokens for a given model. Defaults to inf.
  - Integer = Integer
  - MaxOutputTokens = :inf
    - :inf
- metadata: Metadata
  
  Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard.
  
  Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.
- output_modalities: Array[:text | :audio]
  
  The set of modalities the model used to respond, currently the only possible values are [\"audio\"], [\"text\"]. Audio output always include a text transcript. Setting the output to mode text will disable audio output from the model.
  - :text
  - :audio
- parallel_tool_calls: bool
  
  Whether the model may call multiple tools in parallel. Only supported by reasoning Realtime models such as gpt-realtime-2.
- prompt: ResponsePrompt
  
  Reference to a prompt template and its variables. Learn more.
  - id: String
    
    The unique identifier of the prompt template to use.
  - variables: Hash[Symbol, String | ResponseInputText | ResponseInputImage | ResponseInputFile]
    
    Optional map of values to substitute in for variables in your prompt. The substitution values can either be strings, or other Response input types like images or files.
    - String = String
    - class ResponseInputText
      
      A text input to the model.
      - text: String
        
        The text input to the model.
      - type: :input_text
        
        The type of the input item. Always input_text.
        
        :input_text
    - class ResponseInputImage
      
      An image input to the model. Learn about image inputs.
      - detail: :low | :high | :auto | :original
        
        The detail level of the image to be sent to the model. One of high, low, auto, or original. Defaults to auto.
        
        :low
        
        :high
        
        :auto
        
        :original
      - type: :input_image
        
        The type of the input item. Always input_image.
        
        :input_image
      - file_id: String
        
        The ID of the file to be sent to the model.
      - image_url: String
        
        The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.
    - class ResponseInputFile
      
      A file input to the model.
      - type: :input_file
        
        The type of the input item. Always input_file.
        
        :input_file
      - detail: :low | :high
        
        The detail level of the file to be sent to the model. Use low for the default rendering behavior, or high to render the file at higher quality. Defaults to low.
        
        :low
        
        :high
      - file_data: String
        
        The content of the file to be sent to the model.
      - file_id: String
        
        The ID of the file to be sent to the model.
      - file_url: String
        
        The URL of the file to be sent to the model.
      - filename: String
        
        The name of the file to be sent to the model.
  - version: String
    
    Optional version of the prompt template.
- reasoning: RealtimeReasoning
  
  Configuration for reasoning-capable Realtime models such as gpt-realtime-2.
  - effort: RealtimeReasoningEffort
    
    Constrains effort on reasoning for reasoning-capable Realtime models such as gpt-realtime-2.
    - :minimal
    - :low
    - :medium
    - :high
    - :xhigh
- tool_choice: ToolChoiceOptions | ToolChoiceFunction | ToolChoiceMcp
  
  How the model chooses tools. Provide one of the string modes or force a specific function/MCP tool.
  - ToolChoiceOptions = :none | :auto | :required
    
    Controls which (if any) tool is called by the model.
    
    none means the model will not call any tool and instead generates a message.
    
    auto means the model can pick between generating a message or calling one or more tools.
    
    required means the model must call one or more tools.
    - :none
    - :auto
    - :required
  - class ToolChoiceFunction
    
    Use this option to force the model to call a specific function.
    - name: String
      
      The name of the function to call.
    - type: :function
      
      For function calling, the type is always function.
      - :function
  - class ToolChoiceMcp
    
    Use this option to force the model to call a specific tool on a remote MCP server.
    - server_label: String
      
      The label of the MCP server to use.
    - type: :mcp
      
      For MCP tools, the type is always mcp.
      - :mcp
    - name: String
      
      The name of the tool to call on the server.
- tools: Array[RealtimeFunctionTool | RealtimeResponseCreateMcpTool]
  
  Tools available to the model.
  - class RealtimeFunctionTool
    - description: String
      
      The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything).
    - name: String
      
      The name of the function.
    - parameters: untyped
      
      Parameters of the function in JSON Schema.
    - type: :function
      
      The type of the tool, i.e. function.
      - :function
  - class RealtimeResponseCreateMcpTool
    
    Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.
    - server_label: String
      
      A label for this MCP server, used to identify it in tool calls.
    - type: :mcp
      
      The type of the MCP tool. Always mcp.
      - :mcp
    - allowed_tools: Array[String] | McpToolFilter{ read_only, tool_names}
      
      List of allowed tool names or a filter object.
      - McpAllowedTools = Array[String]
        
        A string array of allowed tool names
      - class McpToolFilter
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
    - authorization: String
      
      An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.
    - connector_id: :connector_dropbox | :connector_gmail | :connector_googlecalendar | 5 more
      
      Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here.
      
      Currently supported connector_id values are:
      - Dropbox: connector_dropbox
      - Gmail: connector_gmail
      - Google Calendar: connector_googlecalendar
      - Google Drive: connector_googledrive
      - Microsoft Teams: connector_microsoftteams
      - Outlook Calendar: connector_outlookcalendar
      - Outlook Email: connector_outlookemail
      - SharePoint: connector_sharepoint
      - :connector_dropbox
      - :connector_gmail
      - :connector_googlecalendar
      - :connector_googledrive
      - :connector_microsoftteams
      - :connector_outlookcalendar
      - :connector_outlookemail
      - :connector_sharepoint
    - defer_loading: bool
      
      Whether this MCP tool is deferred and discovered via tool search.
    - headers: Hash[Symbol, String]
      
      Optional HTTP headers to send to the MCP server. Use for authentication or other purposes.
    - require_approval: McpToolApprovalFilter{ always, never} | :always | :never
      
      Specify which of the MCP server's tools require approval.
      - class McpToolApprovalFilter
        
        Specify which of the MCP server's tools require approval. Can be always, never, or a filter object associated with tools that require approval.
        
        always: Always{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        never: Never{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
      - McpToolApprovalSetting = :always | :never
        
        Specify a single approval policy for all tools. One of always or never. When set to always, all tools will require approval. When set to never, all tools will not require approval.
        
        :always
        
        :never
    - server_description: String
      
      Optional description of the MCP server, used to provide more context.
    - server_url: String
      
      The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided.
    - tunnel_id: String
      
      The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided.

Realtime Response Status

class RealtimeResponseStatus

Additional details about the status.
- error: Error{ code, type}
  
  A description of the error that caused the response to fail, populated when the status is failed.
  - code: String
    
    Error code, if any.
  - type: String
    
    The type of error.
- reason: :turn_detected | :client_cancelled | :max_output_tokens | :content_filter
  
  The reason the Response did not complete. For a cancelled Response, one of turn_detected (the server VAD detected a new start of speech) or client_cancelled (the client sent a cancel event). For an incomplete Response, one of max_output_tokens or content_filter (the server-side safety filter activated and cut off the response).
  - :turn_detected
  - :client_cancelled
  - :max_output_tokens
  - :content_filter
- type: :completed | :cancelled | :incomplete | :failed
  
  The type of error that caused the response to fail, corresponding with the status field (completed, cancelled, incomplete, failed).
  - :completed
  - :cancelled
  - :incomplete
  - :failed

Realtime Response Usage

class RealtimeResponseUsage

Usage statistics for the Response, this will correspond to billing. A Realtime API session will maintain a conversation context and append new Items to the Conversation, thus output from previous turns (text and audio tokens) will become the input for later turns.
- input_token_details: RealtimeResponseUsageInputTokenDetails
  
  Details about the input tokens used in the Response. Cached tokens are tokens from previous turns in the conversation that are included as context for the current response. Cached tokens here are counted as a subset of input tokens, meaning input tokens will include cached and uncached tokens.
  - audio_tokens: Integer
    
    The number of audio tokens used as input for the Response.
  - cached_tokens: Integer
    
    The number of cached tokens used as input for the Response.
  - cached_tokens_details: CachedTokensDetails{ audio_tokens, image_tokens, text_tokens}
    
    Details about the cached tokens used as input for the Response.
    - audio_tokens: Integer
      
      The number of cached audio tokens used as input for the Response.
    - image_tokens: Integer
      
      The number of cached image tokens used as input for the Response.
    - text_tokens: Integer
      
      The number of cached text tokens used as input for the Response.
  - image_tokens: Integer
    
    The number of image tokens used as input for the Response.
  - text_tokens: Integer
    
    The number of text tokens used as input for the Response.
- input_tokens: Integer
  
  The number of input tokens used in the Response, including text and audio tokens.
- output_token_details: RealtimeResponseUsageOutputTokenDetails
  
  Details about the output tokens used in the Response.
  - audio_tokens: Integer
    
    The number of audio tokens used in the Response.
  - text_tokens: Integer
    
    The number of text tokens used in the Response.
- output_tokens: Integer
  
  The number of output tokens sent in the Response, including text and audio tokens.
- total_tokens: Integer
  
  The total number of tokens in the Response including input and output text and audio tokens.

Realtime Response Usage Input Token Details

class RealtimeResponseUsageInputTokenDetails

Details about the input tokens used in the Response. Cached tokens are tokens from previous turns in the conversation that are included as context for the current response. Cached tokens here are counted as a subset of input tokens, meaning input tokens will include cached and uncached tokens.
- audio_tokens: Integer
  
  The number of audio tokens used as input for the Response.
- cached_tokens: Integer
  
  The number of cached tokens used as input for the Response.
- cached_tokens_details: CachedTokensDetails{ audio_tokens, image_tokens, text_tokens}
  
  Details about the cached tokens used as input for the Response.
  - audio_tokens: Integer
    
    The number of cached audio tokens used as input for the Response.
  - image_tokens: Integer
    
    The number of cached image tokens used as input for the Response.
  - text_tokens: Integer
    
    The number of cached text tokens used as input for the Response.
- image_tokens: Integer
  
  The number of image tokens used as input for the Response.
- text_tokens: Integer
  
  The number of text tokens used as input for the Response.

Realtime Response Usage Output Token Details

class RealtimeResponseUsageOutputTokenDetails

Details about the output tokens used in the Response.
- audio_tokens: Integer
  
  The number of audio tokens used in the Response.
- text_tokens: Integer
  
  The number of text tokens used in the Response.

Realtime Server Event

RealtimeServerEvent = ConversationCreatedEvent | ConversationItemCreatedEvent | ConversationItemDeletedEvent | 43 more

A realtime server event.
- class ConversationCreatedEvent
  
  Returned when a conversation is created. Emitted right after session creation.
  - conversation: Conversation{ id, object}
    
    The conversation resource.
    - id: String
      
      The unique ID of the conversation.
    - object: :"realtime.conversation"
      
      The object type, must be realtime.conversation.
      - :"realtime.conversation"
  - event_id: String
    
    The unique ID of the server event.
  - type: :"conversation.created"
    
    The event type, must be conversation.created.
    - :"conversation.created"
- class ConversationItemCreatedEvent
  
  Returned when a conversation item is created. There are several scenarios that produce this event:
  - The server is generating a Response, which if successful will produce either one or two Items, which will be of type message (role assistant) or type function_call.
  - The input audio buffer has been committed, either by the client or the server (in server_vad mode). The server will take the content of the input audio buffer and add it to a new user message Item.
  - The client has sent a conversation.item.create event to add a new Item to the Conversation.
  - event_id: String
    
    The unique ID of the server event.
  - item: ConversationItem
    
    A single item within a Realtime conversation.
    - class RealtimeConversationItemSystemMessage
      
      A system message in a Realtime conversation can be used to provide additional context or instructions to the model. This is similar but distinct from the instruction prompt provided at the start of a conversation, as system messages can be added at any point in the conversation. For major changes to the conversation's behavior, use instructions, but for smaller updates (e.g. "the user is now asking about a different topic"), use system messages.
      - content: Array[Content{ text, type}]
        
        The content of the message.
        
        text: String
        
        The text content.
        
        type: :input_text
        
        The content type. Always input_text for system messages.
        
        :input_text
      - role: :system
        
        The role of the message sender. Always system.
        
        :system
      - type: :message
        
        The type of the item. Always message.
        
        :message
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeConversationItemUserMessage
      
      A user message item in a Realtime conversation.
      - content: Array[Content{ audio, detail, image_url, 3 more}]
        
        The content of the message.
        
        audio: String
        
        Base64-encoded audio bytes (for input_audio), these will be parsed as the format specified in the session input audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
        
        detail: :auto | :low | :high
        
        The detail level of the image (for input_image). auto will default to high.
        
        :auto
        
        :low
        
        :high
        
        image_url: String
        
        Base64-encoded image bytes (for input_image) as a data URI. For example data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA.... Supported formats are PNG and JPEG.
        
        text: String
        
        The text content (for input_text).
        
        transcript: String
        
        Transcript of the audio (for input_audio). This is not sent to the model, but will be attached to the message item for reference.
        
        type: :input_text | :input_audio | :input_image
        
        The content type (input_text, input_audio, or input_image).
        
        :input_text
        
        :input_audio
        
        :input_image
      - role: :user
        
        The role of the message sender. Always user.
        
        :user
      - type: :message
        
        The type of the item. Always message.
        
        :message
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeConversationItemAssistantMessage
      
      An assistant message item in a Realtime conversation.
      - content: Array[Content{ audio, text, transcript, type}]
        
        The content of the message.
        
        audio: String
        
        Base64-encoded audio bytes, these will be parsed as the format specified in the session output audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
        
        text: String
        
        The text content.
        
        transcript: String
        
        The transcript of the audio content, this will always be present if the output type is audio.
        
        type: :output_text | :output_audio
        
        The content type, output_text or output_audio depending on the session output_modalities configuration.
        
        :output_text
        
        :output_audio
      - role: :assistant
        
        The role of the message sender. Always assistant.
        
        :assistant
      - type: :message
        
        The type of the item. Always message.
        
        :message
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeConversationItemFunctionCall
      
      A function call item in a Realtime conversation.
      - arguments: String
        
        The arguments of the function call. This is a JSON-encoded string representing the arguments passed to the function, for example {"arg1": "value1", "arg2": 42}.
      - name: String
        
        The name of the function being called.
      - type: :function_call
        
        The type of the item. Always function_call.
        
        :function_call
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - call_id: String
        
        The ID of the function call.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeConversationItemFunctionCallOutput
      
      A function call output item in a Realtime conversation.
      - call_id: String
        
        The ID of the function call this output is for.
      - output: String
        
        The output of the function call, this is free text and can contain any information or simply be empty.
      - type: :function_call_output
        
        The type of the item. Always function_call_output.
        
        :function_call_output
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeMcpApprovalResponse
      
      A Realtime item responding to an MCP approval request.
      - id: String
        
        The unique ID of the approval response.
      - approval_request_id: String
        
        The ID of the approval request being answered.
      - approve: bool
        
        Whether the request was approved.
      - type: :mcp_approval_response
        
        The type of the item. Always mcp_approval_response.
        
        :mcp_approval_response
      - reason: String
        
        Optional reason for the decision.
    - class RealtimeMcpListTools
      
      A Realtime item listing tools available on an MCP server.
      - server_label: String
        
        The label of the MCP server.
      - tools: Array[Tool{ input_schema, name, annotations, description}]
        
        The tools available on the server.
        
        input_schema: untyped
        
        The JSON schema describing the tool's input.
        
        name: String
        
        The name of the tool.
        
        annotations: untyped
        
        Additional annotations about the tool.
        
        description: String
        
        The description of the tool.
      - type: :mcp_list_tools
        
        The type of the item. Always mcp_list_tools.
        
        :mcp_list_tools
      - id: String
        
        The unique ID of the list.
    - class RealtimeMcpToolCall
      
      A Realtime item representing an invocation of a tool on an MCP server.
      - id: String
        
        The unique ID of the tool call.
      - arguments: String
        
        A JSON string of the arguments passed to the tool.
      - name: String
        
        The name of the tool that was run.
      - server_label: String
        
        The label of the MCP server running the tool.
      - type: :mcp_call
        
        The type of the item. Always mcp_call.
        
        :mcp_call
      - approval_request_id: String
        
        The ID of an associated approval request, if any.
      - error: RealtimeMcpProtocolError | RealtimeMcpToolExecutionError | RealtimeMcphttpError
        
        The error from the tool call, if any.
        
        class RealtimeMcpProtocolError
        
        code: Integer
        
        message: String
        
        type: :protocol_error
        
        :protocol_error
        
        class RealtimeMcpToolExecutionError
        
        message: String
        
        type: :tool_execution_error
        
        :tool_execution_error
        
        class RealtimeMcphttpError
        
        code: Integer
        
        message: String
        
        type: :http_error
        
        :http_error
      - output: String
        
        The output from the tool call.
    - class RealtimeMcpApprovalRequest
      
      A Realtime item requesting human approval of a tool invocation.
      - id: String
        
        The unique ID of the approval request.
      - arguments: String
        
        A JSON string of arguments for the tool.
      - name: String
        
        The name of the tool to run.
      - server_label: String
        
        The label of the MCP server making the request.
      - type: :mcp_approval_request
        
        The type of the item. Always mcp_approval_request.
        
        :mcp_approval_request
  - type: :"conversation.item.created"
    
    The event type, must be conversation.item.created.
    - :"conversation.item.created"
  - previous_item_id: String
    
    The ID of the preceding item in the Conversation context, allows the client to understand the order of the conversation. Can be null if the item has no predecessor.
- class ConversationItemDeletedEvent
  
  Returned when an item in the conversation is deleted by the client with a conversation.item.delete event. This event is used to synchronize the server's understanding of the conversation history with the client's view.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the item that was deleted.
  - type: :"conversation.item.deleted"
    
    The event type, must be conversation.item.deleted.
    - :"conversation.item.deleted"
- class ConversationItemInputAudioTranscriptionCompletedEvent
  
  This event is the output of audio transcription for user audio written to the user audio buffer. Transcription begins when the input audio buffer is committed by the client or server (when VAD is enabled). Transcription runs asynchronously with Response creation, so this event may come before or after the Response events.
  
  Realtime API models accept audio natively, and thus input transcription is a separate process run on a separate ASR (Automatic Speech Recognition) model. The transcript may diverge somewhat from the model's interpretation, and should be treated as a rough guide.
  - content_index: Integer
    
    The index of the content part containing the audio.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the item containing the audio that is being transcribed.
  - transcript: String
    
    The transcribed text.
  - type: :"conversation.item.input_audio_transcription.completed"
    
    The event type, must be conversation.item.input_audio_transcription.completed.
    - :"conversation.item.input_audio_transcription.completed"
  - usage: TranscriptTextUsageTokens{ input_tokens, output_tokens, total_tokens, 2 more} | TranscriptTextUsageDuration{ seconds, type}
    
    Usage statistics for the transcription, this is billed according to the ASR model's pricing rather than the realtime model's pricing.
    - class TranscriptTextUsageTokens
      
      Usage statistics for models billed by token usage.
      - input_tokens: Integer
        
        Number of input tokens billed for this request.
      - output_tokens: Integer
        
        Number of output tokens generated.
      - total_tokens: Integer
        
        Total number of tokens used (input + output).
      - type: :tokens
        
        The type of the usage object. Always tokens for this variant.
        
        :tokens
      - input_token_details: InputTokenDetails{ audio_tokens, text_tokens}
        
        Details about the input tokens billed for this request.
        
        audio_tokens: Integer
        
        Number of audio tokens billed for this request.
        
        text_tokens: Integer
        
        Number of text tokens billed for this request.
    - class TranscriptTextUsageDuration
      
      Usage statistics for models billed by audio input duration.
      - seconds: Float
        
        Duration of the input audio in seconds.
      - type: :duration
        
        The type of the usage object. Always duration for this variant.
        
        :duration
  - logprobs: Array[LogProbProperties]
    
    The log probabilities of the transcription.
    - token: String
      
      The token that was used to generate the log probability.
    - bytes: Array[Integer]
      
      The bytes that were used to generate the log probability.
    - logprob: Float
      
      The log probability of the token.
- class ConversationItemInputAudioTranscriptionDeltaEvent
  
  Returned when the text value of an input audio transcription content part is updated with incremental transcription results.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the item containing the audio that is being transcribed.
  - type: :"conversation.item.input_audio_transcription.delta"
    
    The event type, must be conversation.item.input_audio_transcription.delta.
    - :"conversation.item.input_audio_transcription.delta"
  - content_index: Integer
    
    The index of the content part in the item's content array.
  - delta: String
    
    The text delta.
  - logprobs: Array[LogProbProperties]
    
    The log probabilities of the transcription. These can be enabled by configurating the session with "include": ["item.input_audio_transcription.logprobs"]. Each entry in the array corresponds a log probability of which token would be selected for this chunk of transcription. This can help to identify if it was possible there were multiple valid options for a given chunk of transcription.
    - token: String
      
      The token that was used to generate the log probability.
    - bytes: Array[Integer]
      
      The bytes that were used to generate the log probability.
    - logprob: Float
      
      The log probability of the token.
- class ConversationItemInputAudioTranscriptionFailedEvent
  
  Returned when input audio transcription is configured, and a transcription request for a user message failed. These events are separate from other error events so that the client can identify the related Item.
  - content_index: Integer
    
    The index of the content part containing the audio.
  - error: Error{ code, message, param, type}
    
    Details of the transcription error.
    - code: String
      
      Error code, if any.
    - message: String
      
      A human-readable error message.
    - param: String
      
      Parameter related to the error, if any.
    - type: String
      
      The type of error.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the user message item.
  - type: :"conversation.item.input_audio_transcription.failed"
    
    The event type, must be conversation.item.input_audio_transcription.failed.
    - :"conversation.item.input_audio_transcription.failed"
- class ConversationItemRetrieved
  
  Returned when a conversation item is retrieved with conversation.item.retrieve. This is provided as a way to fetch the server's representation of an item, for example to get access to the post-processed audio data after noise cancellation and VAD. It includes the full content of the Item, including audio data.
  - event_id: String
    
    The unique ID of the server event.
  - item: ConversationItem
    
    A single item within a Realtime conversation.
  - type: :"conversation.item.retrieved"
    
    The event type, must be conversation.item.retrieved.
    - :"conversation.item.retrieved"
- class ConversationItemTruncatedEvent
  
  Returned when an earlier assistant audio message item is truncated by the client with a conversation.item.truncate event. This event is used to synchronize the server's understanding of the audio with the client's playback.
  
  This action will truncate the audio and remove the server-side text transcript to ensure there is no text in the context that hasn't been heard by the user.
  - audio_end_ms: Integer
    
    The duration up to which the audio was truncated, in milliseconds.
  - content_index: Integer
    
    The index of the content part that was truncated.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the assistant message item that was truncated.
  - type: :"conversation.item.truncated"
    
    The event type, must be conversation.item.truncated.
    - :"conversation.item.truncated"
- class RealtimeErrorEvent
  
  Returned when an error occurs, which could be a client problem or a server problem. Most errors are recoverable and the session will stay open, we recommend to implementors to monitor and log error messages by default.
  - error: RealtimeError
    
    Details of the error.
    - message: String
      
      A human-readable error message.
    - type: String
      
      The type of error (e.g., "invalid_request_error", "server_error").
    - code: String
      
      Error code, if any.
    - event_id: String
      
      The event_id of the client event that caused the error, if applicable.
    - param: String
      
      Parameter related to the error, if any.
  - event_id: String
    
    The unique ID of the server event.
  - type: :error
    
    The event type, must be error.
    - :error
- class InputAudioBufferClearedEvent
  
  Returned when the input audio buffer is cleared by the client with a input_audio_buffer.clear event.
  - event_id: String
    
    The unique ID of the server event.
  - type: :"input_audio_buffer.cleared"
    
    The event type, must be input_audio_buffer.cleared.
    - :"input_audio_buffer.cleared"
- class InputAudioBufferCommittedEvent
  
  Returned when an input audio buffer is committed, either by the client or automatically in server VAD mode. The item_id property is the ID of the user message item that will be created, thus a conversation.item.created event will also be sent to the client.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the user message item that will be created.
  - type: :"input_audio_buffer.committed"
    
    The event type, must be input_audio_buffer.committed.
    - :"input_audio_buffer.committed"
  - previous_item_id: String
    
    The ID of the preceding item after which the new item will be inserted. Can be null if the item has no predecessor.
- class InputAudioBufferDtmfEventReceivedEvent
  
  SIP Only: Returned when an DTMF event is received. A DTMF event is a message that represents a telephone keypad press (0–9, *, #, A–D). The event property is the keypad that the user press. The received_at is the UTC Unix Timestamp that the server received the event.
  - event: String
    
    The telephone keypad that was pressed by the user.
  - received_at: Integer
    
    UTC Unix Timestamp when DTMF Event was received by server.
  - type: :"input_audio_buffer.dtmf_event_received"
    
    The event type, must be input_audio_buffer.dtmf_event_received.
    - :"input_audio_buffer.dtmf_event_received"
- class InputAudioBufferSpeechStartedEvent
  
  Sent by the server when in server_vad mode to indicate that speech has been detected in the audio buffer. This can happen any time audio is added to the buffer (unless speech is already detected). The client may want to use this event to interrupt audio playback or provide visual feedback to the user.
  
  The client should expect to receive a input_audio_buffer.speech_stopped event when speech stops. The item_id property is the ID of the user message item that will be created when speech stops and will also be included in the input_audio_buffer.speech_stopped event (unless the client manually commits the audio buffer during VAD activation).
  - audio_start_ms: Integer
    
    Milliseconds from the start of all audio written to the buffer during the session when speech was first detected. This will correspond to the beginning of audio sent to the model, and thus includes the prefix_padding_ms configured in the Session.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the user message item that will be created when speech stops.
  - type: :"input_audio_buffer.speech_started"
    
    The event type, must be input_audio_buffer.speech_started.
    - :"input_audio_buffer.speech_started"
- class InputAudioBufferSpeechStoppedEvent
  
  Returned in server_vad mode when the server detects the end of speech in the audio buffer. The server will also send an conversation.item.created event with the user message item that is created from the audio buffer.
  - audio_end_ms: Integer
    
    Milliseconds since the session started when speech stopped. This will correspond to the end of audio sent to the model, and thus includes the min_silence_duration_ms configured in the Session.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the user message item that will be created.
  - type: :"input_audio_buffer.speech_stopped"
    
    The event type, must be input_audio_buffer.speech_stopped.
    - :"input_audio_buffer.speech_stopped"
- class RateLimitsUpdatedEvent
  
  Emitted at the beginning of a Response to indicate the updated rate limits. When a Response is created some tokens will be "reserved" for the output tokens, the rate limits shown here reflect that reservation, which is then adjusted accordingly once the Response is completed.
  - event_id: String
    
    The unique ID of the server event.
  - rate_limits: Array[RateLimit{ limit, name, remaining, reset_seconds}]
    
    List of rate limit information.
    - limit: Integer
      
      The maximum allowed value for the rate limit.
    - name: :requests | :tokens
      
      The name of the rate limit (requests, tokens).
      - :requests
      - :tokens
    - remaining: Integer
      
      The remaining value before the limit is reached.
    - reset_seconds: Float
      
      Seconds until the rate limit resets.
  - type: :"rate_limits.updated"
    
    The event type, must be rate_limits.updated.
    - :"rate_limits.updated"
- class ResponseAudioDeltaEvent
  
  Returned when the model-generated audio is updated.
  - content_index: Integer
    
    The index of the content part in the item's content array.
  - delta: String
    
    Base64-encoded audio data delta.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the item.
  - output_index: Integer
    
    The index of the output item in the response.
  - response_id: String
    
    The ID of the response.
  - type: :"response.output_audio.delta"
    
    The event type, must be response.output_audio.delta.
    - :"response.output_audio.delta"
- class ResponseAudioDoneEvent
  
  Returned when the model-generated audio is done. Also emitted when a Response is interrupted, incomplete, or cancelled.
  - content_index: Integer
    
    The index of the content part in the item's content array.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the item.
  - output_index: Integer
    
    The index of the output item in the response.
  - response_id: String
    
    The ID of the response.
  - type: :"response.output_audio.done"
    
    The event type, must be response.output_audio.done.
    - :"response.output_audio.done"
- class ResponseAudioTranscriptDeltaEvent
  
  Returned when the model-generated transcription of audio output is updated.
  - content_index: Integer
    
    The index of the content part in the item's content array.
  - delta: String
    
    The transcript delta.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the item.
  - output_index: Integer
    
    The index of the output item in the response.
  - response_id: String
    
    The ID of the response.
  - type: :"response.output_audio_transcript.delta"
    
    The event type, must be response.output_audio_transcript.delta.
    - :"response.output_audio_transcript.delta"
- class ResponseAudioTranscriptDoneEvent
  
  Returned when the model-generated transcription of audio output is done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.
  - content_index: Integer
    
    The index of the content part in the item's content array.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the item.
  - output_index: Integer
    
    The index of the output item in the response.
  - response_id: String
    
    The ID of the response.
  - transcript: String
    
    The final transcript of the audio.
  - type: :"response.output_audio_transcript.done"
    
    The event type, must be response.output_audio_transcript.done.
    - :"response.output_audio_transcript.done"
- class ResponseContentPartAddedEvent
  
  Returned when a new content part is added to an assistant message item during response generation.
  - content_index: Integer
    
    The index of the content part in the item's content array.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the item to which the content part was added.
  - output_index: Integer
    
    The index of the output item in the response.
  - part: Part{ audio, text, transcript, type}
    
    The content part that was added.
    - audio: String
      
      Base64-encoded audio data (if type is "audio").
    - text: String
      
      The text content (if type is "text").
    - transcript: String
      
      The transcript of the audio (if type is "audio").
    - type: :text | :audio
      
      The content type ("text", "audio").
      - :text
      - :audio
  - response_id: String
    
    The ID of the response.
  - type: :"response.content_part.added"
    
    The event type, must be response.content_part.added.
    - :"response.content_part.added"
- class ResponseContentPartDoneEvent
  
  Returned when a content part is done streaming in an assistant message item. Also emitted when a Response is interrupted, incomplete, or cancelled.
  - content_index: Integer
    
    The index of the content part in the item's content array.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the item.
  - output_index: Integer
    
    The index of the output item in the response.
  - part: Part{ audio, text, transcript, type}
    
    The content part that is done.
    - audio: String
      
      Base64-encoded audio data (if type is "audio").
    - text: String
      
      The text content (if type is "text").
    - transcript: String
      
      The transcript of the audio (if type is "audio").
    - type: :text | :audio
      
      The content type ("text", "audio").
      - :text
      - :audio
  - response_id: String
    
    The ID of the response.
  - type: :"response.content_part.done"
    
    The event type, must be response.content_part.done.
    - :"response.content_part.done"
- class ResponseCreatedEvent
  
  Returned when a new Response is created. The first event of response creation, where the response is in an initial state of in_progress.
  - event_id: String
    
    The unique ID of the server event.
  - response: RealtimeResponse
    
    The response resource.
    - id: String
      
      The unique ID of the response, will look like resp_1234.
    - audio: Audio{ output}
      
      Configuration for audio output.
      - output: Output{ format_, voice}
        
        format_: RealtimeAudioFormats
        
        The format of the output audio.
        
        class AudioPCM
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
        
        type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
        
        class AudioPCMU
        
        The G.711 μ-law format.
        
        type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
        
        class AudioPCMA
        
        The G.711 A-law format.
        
        type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
        
        voice: String | :alloy | :ash | :ballad | 7 more
        
        The voice the model uses to respond. Voice cannot be changed during the session once the model has responded with audio at least once. Current voice options are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. We recommend marin and cedar for best quality.
        
        String = String
        
        Voice = :alloy | :ash | :ballad | 7 more
        
        The voice the model uses to respond. Voice cannot be changed during the session once the model has responded with audio at least once. Current voice options are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. We recommend marin and cedar for best quality.
        
        :alloy
        
        :ash
        
        :ballad
        
        :coral
        
        :echo
        
        :sage
        
        :shimmer
        
        :verse
        
        :marin
        
        :cedar
    - conversation_id: String
      
      Which conversation the response is added to, determined by the conversation field in the response.create event. If auto, the response will be added to the default conversation and the value of conversation_id will be an id like conv_1234. If none, the response will not be added to any conversation and the value of conversation_id will be null. If responses are being triggered automatically by VAD the response will be added to the default conversation
    - max_output_tokens: Integer | :inf
      
      Maximum number of output tokens for a single assistant response, inclusive of tool calls, that was used in this response.
      - Integer = Integer
      - MaxOutputTokens = :inf
        
        :inf
    - metadata: Metadata
      
      Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard.
      
      Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.
    - object: :"realtime.response"
      
      The object type, must be realtime.response.
      - :"realtime.response"
    - output: Array[ConversationItem]
      
      The list of output items generated by the response.
      - class RealtimeConversationItemSystemMessage
        
        A system message in a Realtime conversation can be used to provide additional context or instructions to the model. This is similar but distinct from the instruction prompt provided at the start of a conversation, as system messages can be added at any point in the conversation. For major changes to the conversation's behavior, use instructions, but for smaller updates (e.g. "the user is now asking about a different topic"), use system messages.
      - class RealtimeConversationItemUserMessage
        
        A user message item in a Realtime conversation.
      - class RealtimeConversationItemAssistantMessage
        
        An assistant message item in a Realtime conversation.
      - class RealtimeConversationItemFunctionCall
        
        A function call item in a Realtime conversation.
      - class RealtimeConversationItemFunctionCallOutput
        
        A function call output item in a Realtime conversation.
      - class RealtimeMcpApprovalResponse
        
        A Realtime item responding to an MCP approval request.
      - class RealtimeMcpListTools
        
        A Realtime item listing tools available on an MCP server.
      - class RealtimeMcpToolCall
        
        A Realtime item representing an invocation of a tool on an MCP server.
      - class RealtimeMcpApprovalRequest
        
        A Realtime item requesting human approval of a tool invocation.
    - output_modalities: Array[:text | :audio]
      
      The set of modalities the model used to respond, currently the only possible values are [\"audio\"], [\"text\"]. Audio output always include a text transcript. Setting the output to mode text will disable audio output from the model.
      - :text
      - :audio
    - status: :completed | :cancelled | :failed | 2 more
      
      The final status of the response (completed, cancelled, failed, or incomplete, in_progress).
      - :completed
      - :cancelled
      - :failed
      - :incomplete
      - :in_progress
    - status_details: RealtimeResponseStatus
      
      Additional details about the status.
      - error: Error{ code, type}
        
        A description of the error that caused the response to fail, populated when the status is failed.
        
        code: String
        
        Error code, if any.
        
        type: String
        
        The type of error.
      - reason: :turn_detected | :client_cancelled | :max_output_tokens | :content_filter
        
        The reason the Response did not complete. For a cancelled Response, one of turn_detected (the server VAD detected a new start of speech) or client_cancelled (the client sent a cancel event). For an incomplete Response, one of max_output_tokens or content_filter (the server-side safety filter activated and cut off the response).
        
        :turn_detected
        
        :client_cancelled
        
        :max_output_tokens
        
        :content_filter
      - type: :completed | :cancelled | :incomplete | :failed
        
        The type of error that caused the response to fail, corresponding with the status field (completed, cancelled, incomplete, failed).
        
        :completed
        
        :cancelled
        
        :incomplete
        
        :failed
    - usage: RealtimeResponseUsage
      
      Usage statistics for the Response, this will correspond to billing. A Realtime API session will maintain a conversation context and append new Items to the Conversation, thus output from previous turns (text and audio tokens) will become the input for later turns.
      - input_token_details: RealtimeResponseUsageInputTokenDetails
        
        Details about the input tokens used in the Response. Cached tokens are tokens from previous turns in the conversation that are included as context for the current response. Cached tokens here are counted as a subset of input tokens, meaning input tokens will include cached and uncached tokens.
        
        audio_tokens: Integer
        
        The number of audio tokens used as input for the Response.
        
        cached_tokens: Integer
        
        The number of cached tokens used as input for the Response.
        
        cached_tokens_details: CachedTokensDetails{ audio_tokens, image_tokens, text_tokens}
        
        Details about the cached tokens used as input for the Response.
        
        audio_tokens: Integer
        
        The number of cached audio tokens used as input for the Response.
        
        image_tokens: Integer
        
        The number of cached image tokens used as input for the Response.
        
        text_tokens: Integer
        
        The number of cached text tokens used as input for the Response.
        
        image_tokens: Integer
        
        The number of image tokens used as input for the Response.
        
        text_tokens: Integer
        
        The number of text tokens used as input for the Response.
      - input_tokens: Integer
        
        The number of input tokens used in the Response, including text and audio tokens.
      - output_token_details: RealtimeResponseUsageOutputTokenDetails
        
        Details about the output tokens used in the Response.
        
        audio_tokens: Integer
        
        The number of audio tokens used in the Response.
        
        text_tokens: Integer
        
        The number of text tokens used in the Response.
      - output_tokens: Integer
        
        The number of output tokens sent in the Response, including text and audio tokens.
      - total_tokens: Integer
        
        The total number of tokens in the Response including input and output text and audio tokens.
  - type: :"response.created"
    
    The event type, must be response.created.
    - :"response.created"
- class ResponseDoneEvent
  
  Returned when a Response is done streaming. Always emitted, no matter the final state. The Response object included in the response.done event will include all output Items in the Response but will omit the raw audio data.
  
  Clients should check the status field of the Response to determine if it was successful (completed) or if there was another outcome: cancelled, failed, or incomplete.
  
  A response will contain all output items that were generated during the response, excluding any audio content.
  - event_id: String
    
    The unique ID of the server event.
  - response: RealtimeResponse
    
    The response resource.
  - type: :"response.done"
    
    The event type, must be response.done.
    - :"response.done"
- class ResponseFunctionCallArgumentsDeltaEvent
  
  Returned when the model-generated function call arguments are updated.
  - call_id: String
    
    The ID of the function call.
  - delta: String
    
    The arguments delta as a JSON string.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the function call item.
  - output_index: Integer
    
    The index of the output item in the response.
  - response_id: String
    
    The ID of the response.
  - type: :"response.function_call_arguments.delta"
    
    The event type, must be response.function_call_arguments.delta.
    - :"response.function_call_arguments.delta"
- class ResponseFunctionCallArgumentsDoneEvent
  
  Returned when the model-generated function call arguments are done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.
  - arguments: String
    
    The final arguments as a JSON string.
  - call_id: String
    
    The ID of the function call.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the function call item.
  - name: String
    
    The name of the function that was called.
  - output_index: Integer
    
    The index of the output item in the response.
  - response_id: String
    
    The ID of the response.
  - type: :"response.function_call_arguments.done"
    
    The event type, must be response.function_call_arguments.done.
    - :"response.function_call_arguments.done"
- class ResponseOutputItemAddedEvent
  
  Returned when a new Item is created during Response generation.
  - event_id: String
    
    The unique ID of the server event.
  - item: ConversationItem
    
    A single item within a Realtime conversation.
  - output_index: Integer
    
    The index of the output item in the Response.
  - response_id: String
    
    The ID of the Response to which the item belongs.
  - type: :"response.output_item.added"
    
    The event type, must be response.output_item.added.
    - :"response.output_item.added"
- class ResponseOutputItemDoneEvent
  
  Returned when an Item is done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.
  - event_id: String
    
    The unique ID of the server event.
  - item: ConversationItem
    
    A single item within a Realtime conversation.
  - output_index: Integer
    
    The index of the output item in the Response.
  - response_id: String
    
    The ID of the Response to which the item belongs.
  - type: :"response.output_item.done"
    
    The event type, must be response.output_item.done.
    - :"response.output_item.done"
- class ResponseTextDeltaEvent
  
  Returned when the text value of an "output_text" content part is updated.
  - content_index: Integer
    
    The index of the content part in the item's content array.
  - delta: String
    
    The text delta.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the item.
  - output_index: Integer
    
    The index of the output item in the response.
  - response_id: String
    
    The ID of the response.
  - type: :"response.output_text.delta"
    
    The event type, must be response.output_text.delta.
    - :"response.output_text.delta"
- class ResponseTextDoneEvent
  
  Returned when the text value of an "output_text" content part is done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.
  - content_index: Integer
    
    The index of the content part in the item's content array.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the item.
  - output_index: Integer
    
    The index of the output item in the response.
  - response_id: String
    
    The ID of the response.
  - text: String
    
    The final text content.
  - type: :"response.output_text.done"
    
    The event type, must be response.output_text.done.
    - :"response.output_text.done"
- class SessionCreatedEvent
  
  Returned when a Session is created. Emitted automatically when a new connection is established as the first server event. This event will contain the default Session configuration.
  - event_id: String
    
    The unique ID of the server event.
  - session: RealtimeSessionCreateRequest | RealtimeTranscriptionSessionCreateRequest
    
    The session configuration.
    - class RealtimeSessionCreateRequest
      
      Realtime session object configuration.
      - type: :realtime
        
        The type of session to create. Always realtime for the Realtime API.
        
        :realtime
      - audio: RealtimeAudioConfig
        
        Configuration for input and output audio.
        
        input: RealtimeAudioConfigInput
        
        format_: RealtimeAudioFormats
        
        The format of the input audio.
        
        noise_reduction: NoiseReduction{ type}
        
        Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
        
        transcription: AudioTranscription
        
        Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
        
        delay: :minimal | :low | :medium | 2 more
        
        Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
        
        :minimal
        
        :low
        
        :medium
        
        :high
        
        :xhigh
        
        language: String
        
        The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
        
        model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        String = String
        
        Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        :"whisper-1"
        
        :"gpt-4o-mini-transcribe"
        
        :"gpt-4o-mini-transcribe-2025-12-15"
        
        :"gpt-4o-transcribe"
        
        :"gpt-4o-transcribe-diarize"
        
        :"gpt-realtime-whisper"
        
        prompt: String
        
        An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
        
        turn_detection: RealtimeAudioInputTurnDetection
        
        Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
        
        Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
        
        Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
        
        For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
        
        class ServerVad
        
        Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
        
        type: :server_vad
        
        Type of turn detection, server_vad to turn on simple Server VAD.
        
        :server_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        idle_timeout_ms: Integer
        
        Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
        
        The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
        
        An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        prefix_padding_ms: Integer
        
        Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
        
        silence_duration_ms: Integer
        
        Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
        
        threshold: Float
        
        Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
        
        class SemanticVad
        
        Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
        
        type: :semantic_vad
        
        Type of turn detection, semantic_vad to turn on Semantic VAD.
        
        :semantic_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs.
        
        eagerness: :low | :medium | :high | :auto
        
        Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
        
        :low
        
        :medium
        
        :high
        
        :auto
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.
        
        output: RealtimeAudioConfigOutput
        
        format_: RealtimeAudioFormats
        
        The format of the output audio.
        
        speed: Float
        
        The speed of the model's spoken response as a multiple of the original speed. 1.0 is the default speed. 0.25 is the minimum speed. 1.5 is the maximum speed. This value can only be changed in between model turns, not while a response is in progress.
        
        This parameter is a post-processing adjustment to the audio after it is generated, it's also possible to prompt the model to speak faster or slower.
        
        voice: String | :alloy | :ash | :ballad | 7 more | ID{ id}
        
        The voice the model uses to respond. Supported built-in voices are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. You may also provide a custom voice object with an id, for example { "id": "voice_1234" }. Voice cannot be changed during the session once the model has responded with audio at least once. We recommend marin and cedar for best quality.
        
        String = String
        
        Voice = :alloy | :ash | :ballad | 7 more
        
        :alloy
        
        :ash
        
        :ballad
        
        :coral
        
        :echo
        
        :sage
        
        :shimmer
        
        :verse
        
        :marin
        
        :cedar
        
        class ID
        
        Custom voice reference.
        
        id: String
        
        The custom voice ID, e.g. voice_1234.
      - include: Array[:"item.input_audio_transcription.logprobs"]
        
        Additional fields to include in server outputs.
        
        item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
        
        :"item.input_audio_transcription.logprobs"
      - instructions: String
        
        The default system instructions (i.e. system message) prepended to model calls. This field allows the client to guide the model on desired responses. The model can be instructed on response content and format, (e.g. "be extremely succinct", "act friendly", "here are examples of good responses") and on audio behavior (e.g. "talk quickly", "inject emotion into your voice", "laugh frequently"). The instructions are not guaranteed to be followed by the model, but they provide guidance to the model on the desired behavior.
        
        Note that the server sets default instructions which will be used if this field is not set and are visible in the session.created event at the start of the session.
      - max_output_tokens: Integer | :inf
        
        Maximum number of output tokens for a single assistant response, inclusive of tool calls. Provide an integer between 1 and 4096 to limit output tokens, or inf for the maximum available tokens for a given model. Defaults to inf.
        
        Integer = Integer
        
        MaxOutputTokens = :inf
        
        :inf
      - model: String | :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
        
        The Realtime model used for this session.
        
        String = String
        
        Model = :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
        
        The Realtime model used for this session.
        
        :"gpt-realtime"
        
        :"gpt-realtime-1.5"
        
        :"gpt-realtime-2"
        
        :"gpt-realtime-2025-08-28"
        
        :"gpt-4o-realtime-preview"
        
        :"gpt-4o-realtime-preview-2024-10-01"
        
        :"gpt-4o-realtime-preview-2024-12-17"
        
        :"gpt-4o-realtime-preview-2025-06-03"
        
        :"gpt-4o-mini-realtime-preview"
        
        :"gpt-4o-mini-realtime-preview-2024-12-17"
        
        :"gpt-realtime-mini"
        
        :"gpt-realtime-mini-2025-10-06"
        
        :"gpt-realtime-mini-2025-12-15"
        
        :"gpt-audio-1.5"
        
        :"gpt-audio-mini"
        
        :"gpt-audio-mini-2025-10-06"
        
        :"gpt-audio-mini-2025-12-15"
      - output_modalities: Array[:text | :audio]
        
        The set of modalities the model can respond with. It defaults to ["audio"], indicating that the model will respond with audio plus a transcript. ["text"] can be used to make the model respond with text only. It is not possible to request both text and audio at the same time.
        
        :text
        
        :audio
      - parallel_tool_calls: bool
        
        Whether the model may call multiple tools in parallel. Only supported by reasoning Realtime models such as gpt-realtime-2.
      - prompt: ResponsePrompt
        
        Reference to a prompt template and its variables. Learn more.
        
        id: String
        
        The unique identifier of the prompt template to use.
        
        variables: Hash[Symbol, String | ResponseInputText | ResponseInputImage | ResponseInputFile]
        
        Optional map of values to substitute in for variables in your prompt. The substitution values can either be strings, or other Response input types like images or files.
        
        String = String
        
        class ResponseInputText
        
        A text input to the model.
        
        text: String
        
        The text input to the model.
        
        type: :input_text
        
        The type of the input item. Always input_text.
        
        :input_text
        
        class ResponseInputImage
        
        An image input to the model. Learn about image inputs.
        
        detail: :low | :high | :auto | :original
        
        The detail level of the image to be sent to the model. One of high, low, auto, or original. Defaults to auto.
        
        :low
        
        :high
        
        :auto
        
        :original
        
        type: :input_image
        
        The type of the input item. Always input_image.
        
        :input_image
        
        file_id: String
        
        The ID of the file to be sent to the model.
        
        image_url: String
        
        The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.
        
        class ResponseInputFile
        
        A file input to the model.
        
        type: :input_file
        
        The type of the input item. Always input_file.
        
        :input_file
        
        detail: :low | :high
        
        The detail level of the file to be sent to the model. Use low for the default rendering behavior, or high to render the file at higher quality. Defaults to low.
        
        :low
        
        :high
        
        file_data: String
        
        The content of the file to be sent to the model.
        
        file_id: String
        
        The ID of the file to be sent to the model.
        
        file_url: String
        
        The URL of the file to be sent to the model.
        
        filename: String
        
        The name of the file to be sent to the model.
        
        version: String
        
        Optional version of the prompt template.
      - reasoning: RealtimeReasoning
        
        Configuration for reasoning-capable Realtime models such as gpt-realtime-2.
        
        effort: RealtimeReasoningEffort
        
        Constrains effort on reasoning for reasoning-capable Realtime models such as gpt-realtime-2.
        
        :minimal
        
        :low
        
        :medium
        
        :high
        
        :xhigh
      - tool_choice: RealtimeToolChoiceConfig
        
        How the model chooses tools. Provide one of the string modes or force a specific function/MCP tool.
        
        ToolChoiceOptions = :none | :auto | :required
        
        Controls which (if any) tool is called by the model.
        
        none means the model will not call any tool and instead generates a message.
        
        auto means the model can pick between generating a message or calling one or more tools.
        
        required means the model must call one or more tools.
        
        :none
        
        :auto
        
        :required
        
        class ToolChoiceFunction
        
        Use this option to force the model to call a specific function.
        
        name: String
        
        The name of the function to call.
        
        type: :function
        
        For function calling, the type is always function.
        
        :function
        
        class ToolChoiceMcp
        
        Use this option to force the model to call a specific tool on a remote MCP server.
        
        server_label: String
        
        The label of the MCP server to use.
        
        type: :mcp
        
        For MCP tools, the type is always mcp.
        
        :mcp
        
        name: String
        
        The name of the tool to call on the server.
      - tools: RealtimeToolsConfig
        
        Tools available to the model.
        
        class RealtimeFunctionTool
        
        description: String
        
        The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything).
        
        name: String
        
        The name of the function.
        
        parameters: untyped
        
        Parameters of the function in JSON Schema.
        
        type: :function
        
        The type of the tool, i.e. function.
        
        :function
        
        class Mcp
        
        Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.
        
        server_label: String
        
        A label for this MCP server, used to identify it in tool calls.
        
        type: :mcp
        
        The type of the MCP tool. Always mcp.
        
        :mcp
        
        allowed_tools: Array[String] | McpToolFilter{ read_only, tool_names}
        
        List of allowed tool names or a filter object.
        
        McpAllowedTools = Array[String]
        
        A string array of allowed tool names
        
        class McpToolFilter
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        authorization: String
        
        An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.
        
        connector_id: :connector_dropbox | :connector_gmail | :connector_googlecalendar | 5 more
        
        Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here.
        
        Currently supported connector_id values are:
        
        Dropbox: connector_dropbox
        
        Gmail: connector_gmail
        
        Google Calendar: connector_googlecalendar
        
        Google Drive: connector_googledrive
        
        Microsoft Teams: connector_microsoftteams
        
        Outlook Calendar: connector_outlookcalendar
        
        Outlook Email: connector_outlookemail
        
        SharePoint: connector_sharepoint
        
        :connector_dropbox
        
        :connector_gmail
        
        :connector_googlecalendar
        
        :connector_googledrive
        
        :connector_microsoftteams
        
        :connector_outlookcalendar
        
        :connector_outlookemail
        
        :connector_sharepoint
        
        defer_loading: bool
        
        Whether this MCP tool is deferred and discovered via tool search.
        
        headers: Hash[Symbol, String]
        
        Optional HTTP headers to send to the MCP server. Use for authentication or other purposes.
        
        require_approval: McpToolApprovalFilter{ always, never} | :always | :never
        
        Specify which of the MCP server's tools require approval.
        
        class McpToolApprovalFilter
        
        Specify which of the MCP server's tools require approval. Can be always, never, or a filter object associated with tools that require approval.
        
        always: Always{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        never: Never{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        McpToolApprovalSetting = :always | :never
        
        Specify a single approval policy for all tools. One of always or never. When set to always, all tools will require approval. When set to never, all tools will not require approval.
        
        :always
        
        :never
        
        server_description: String
        
        Optional description of the MCP server, used to provide more context.
        
        server_url: String
        
        The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided.
        
        tunnel_id: String
        
        The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided.
      - tracing: RealtimeTracingConfig
        
        Realtime API can write session traces to the Traces Dashboard. Set to null to disable tracing. Once tracing is enabled for a session, the configuration cannot be modified.
        
        auto will create a trace for the session with default values for the workflow name, group id, and metadata.
        
        RealtimeTracingConfig = :auto
        
        Enables tracing and sets default values for tracing configuration options. Always auto.
        
        :auto
        
        class TracingConfiguration
        
        Granular configuration for tracing.
        
        group_id: String
        
        The group id to attach to this trace to enable filtering and grouping in the Traces Dashboard.
        
        metadata: untyped
        
        The arbitrary metadata to attach to this trace to enable filtering in the Traces Dashboard.
        
        workflow_name: String
        
        The name of the workflow to attach to this trace. This is used to name the trace in the Traces Dashboard.
      - truncation: RealtimeTruncation
        
        When the number of tokens in a conversation exceeds the model's input token limit, the conversation be truncated, meaning messages (starting from the oldest) will not be included in the model's context. A 32k context model with 4,096 max output tokens can only include 28,224 tokens in the context before truncation occurs.
        
        Clients can configure truncation behavior to truncate with a lower max token limit, which is an effective way to control token usage and cost.
        
        Truncation will reduce the number of cached tokens on the next turn (busting the cache), since messages are dropped from the beginning of the context. However, clients can also configure truncation to retain messages up to a fraction of the maximum context size, which will reduce the need for future truncations and thus improve the cache rate.
        
        Truncation can be disabled entirely, which means the server will never truncate but would instead return an error if the conversation exceeds the model's input token limit.
        
        RealtimeTruncationStrategy = :auto | :disabled
        
        The truncation strategy to use for the session. auto is the default truncation strategy. disabled will disable truncation and emit errors when the conversation exceeds the input token limit.
        
        :auto
        
        :disabled
        
        class RealtimeTruncationRetentionRatio
        
        Retain a fraction of the conversation tokens when the conversation exceeds the input token limit. This allows you to amortize truncations across multiple turns, which can help improve cached token usage.
        
        retention_ratio: Float
        
        Fraction of post-instruction conversation tokens to retain (0.0 - 1.0) when the conversation exceeds the input token limit. Setting this to 0.8 means that messages will be dropped until 80% of the maximum allowed tokens are used. This helps reduce the frequency of truncations and improve cache rates.
        
        type: :retention_ratio
        
        Use retention ratio truncation.
        
        :retention_ratio
        
        token_limits: TokenLimits{ post_instructions}
        
        Optional custom token limits for this truncation strategy. If not provided, the model's default token limits will be used.
        
        post_instructions: Integer
        
        Maximum tokens allowed in the conversation after instructions (which including tool definitions). For example, setting this to 5,000 would mean that truncation would occur when the conversation exceeds 5,000 tokens after instructions. This cannot be higher than the model's context window size minus the maximum output tokens.
    - class RealtimeTranscriptionSessionCreateRequest
      
      Realtime transcription session object configuration.
      - type: :transcription
        
        The type of session to create. Always transcription for transcription sessions.
        
        :transcription
      - audio: RealtimeTranscriptionSessionAudio
        
        Configuration for input and output audio.
        
        input: RealtimeTranscriptionSessionAudioInput
        
        format_: RealtimeAudioFormats
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        noise_reduction: NoiseReduction{ type}
        
        Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        transcription: AudioTranscription
        
        Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
        
        turn_detection: RealtimeTranscriptionSessionAudioInputTurnDetection
        
        Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
        
        Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
        
        Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
        
        For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
        
        class ServerVad
        
        Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
        
        type: :server_vad
        
        Type of turn detection, server_vad to turn on simple Server VAD.
        
        :server_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        idle_timeout_ms: Integer
        
        Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
        
        The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
        
        An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        prefix_padding_ms: Integer
        
        Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
        
        silence_duration_ms: Integer
        
        Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
        
        threshold: Float
        
        Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
        
        class SemanticVad
        
        Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
        
        type: :semantic_vad
        
        Type of turn detection, semantic_vad to turn on Semantic VAD.
        
        :semantic_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs.
        
        eagerness: :low | :medium | :high | :auto
        
        Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
        
        :low
        
        :medium
        
        :high
        
        :auto
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.
      - include: Array[:"item.input_audio_transcription.logprobs"]
        
        Additional fields to include in server outputs.
        
        item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
        
        :"item.input_audio_transcription.logprobs"
  - type: :"session.created"
    
    The event type, must be session.created.
    - :"session.created"
- class SessionUpdatedEvent
  
  Returned when a session is updated with a session.update event, unless there is an error.
  - event_id: String
    
    The unique ID of the server event.
  - session: RealtimeSessionCreateRequest | RealtimeTranscriptionSessionCreateRequest
    
    The session configuration.
    - class RealtimeSessionCreateRequest
      
      Realtime session object configuration.
    - class RealtimeTranscriptionSessionCreateRequest
      
      Realtime transcription session object configuration.
  - type: :"session.updated"
    
    The event type, must be session.updated.
    - :"session.updated"
- class OutputAudioBufferStarted
  
  WebRTC/SIP Only: Emitted when the server begins streaming audio to the client. This event is emitted after an audio content part has been added (response.content_part.added) to the response. Learn more.
  - event_id: String
    
    The unique ID of the server event.
  - response_id: String
    
    The unique ID of the response that produced the audio.
  - type: :"output_audio_buffer.started"
    
    The event type, must be output_audio_buffer.started.
    - :"output_audio_buffer.started"
- class OutputAudioBufferStopped
  
  WebRTC/SIP Only: Emitted when the output audio buffer has been completely drained on the server, and no more audio is forthcoming. This event is emitted after the full response data has been sent to the client (response.done). Learn more.
  - event_id: String
    
    The unique ID of the server event.
  - response_id: String
    
    The unique ID of the response that produced the audio.
  - type: :"output_audio_buffer.stopped"
    
    The event type, must be output_audio_buffer.stopped.
    - :"output_audio_buffer.stopped"
- class OutputAudioBufferCleared
  
  WebRTC/SIP Only: Emitted when the output audio buffer is cleared. This happens either in VAD mode when the user has interrupted (input_audio_buffer.speech_started), or when the client has emitted the output_audio_buffer.clear event to manually cut off the current audio response. Learn more.
  - event_id: String
    
    The unique ID of the server event.
  - response_id: String
    
    The unique ID of the response that produced the audio.
  - type: :"output_audio_buffer.cleared"
    
    The event type, must be output_audio_buffer.cleared.
    - :"output_audio_buffer.cleared"
- class ConversationItemAdded
  
  Sent by the server when an Item is added to the default Conversation. This can happen in several cases:
  - When the client sends a conversation.item.create event.
  - When the input audio buffer is committed. In this case the item will be a user message containing the audio from the buffer.
  - When the model is generating a Response. In this case the conversation.item.added event will be sent when the model starts generating a specific Item, and thus it will not yet have any content (and status will be in_progress).
  The event will include the full content of the Item (except when model is generating a Response) except for audio data, which can be retrieved separately with a conversation.item.retrieve event if necessary.
  - event_id: String
    
    The unique ID of the server event.
  - item: ConversationItem
    
    A single item within a Realtime conversation.
  - type: :"conversation.item.added"
    
    The event type, must be conversation.item.added.
    - :"conversation.item.added"
  - previous_item_id: String
    
    The ID of the item that precedes this one, if any. This is used to maintain ordering when items are inserted.
- class ConversationItemDone
  
  Returned when a conversation item is finalized.
  
  The event will include the full content of the Item except for audio data, which can be retrieved separately with a conversation.item.retrieve event if needed.
  - event_id: String
    
    The unique ID of the server event.
  - item: ConversationItem
    
    A single item within a Realtime conversation.
  - type: :"conversation.item.done"
    
    The event type, must be conversation.item.done.
    - :"conversation.item.done"
  - previous_item_id: String
    
    The ID of the item that precedes this one, if any. This is used to maintain ordering when items are inserted.
- class InputAudioBufferTimeoutTriggered
  
  Returned when the Server VAD timeout is triggered for the input audio buffer. This is configured with idle_timeout_ms in the turn_detection settings of the session, and it indicates that there hasn't been any speech detected for the configured duration.
  
  The audio_start_ms and audio_end_ms fields indicate the segment of audio after the last model response up to the triggering time, as an offset from the beginning of audio written to the input audio buffer. This means it demarcates the segment of audio that was silent and the difference between the start and end values will roughly match the configured timeout.
  
  The empty audio will be committed to the conversation as an input_audio item (there will be a input_audio_buffer.committed event) and a model response will be generated. There may be speech that didn't trigger VAD but is still detected by the model, so the model may respond with something relevant to the conversation or a prompt to continue speaking.
  - audio_end_ms: Integer
    
    Millisecond offset of audio written to the input audio buffer at the time the timeout was triggered.
  - audio_start_ms: Integer
    
    Millisecond offset of audio written to the input audio buffer that was after the playback time of the last model response.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the item associated with this segment.
  - type: :"input_audio_buffer.timeout_triggered"
    
    The event type, must be input_audio_buffer.timeout_triggered.
    - :"input_audio_buffer.timeout_triggered"
- class ConversationItemInputAudioTranscriptionSegment
  
  Returned when an input audio transcription segment is identified for an item.
  - id: String
    
    The segment identifier.
  - content_index: Integer
    
    The index of the input audio content part within the item.
  - end_: Float
    
    End time of the segment in seconds.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the item containing the input audio content.
  - speaker: String
    
    The detected speaker label for this segment.
  - start: Float
    
    Start time of the segment in seconds.
  - text: String
    
    The text for this segment.
  - type: :"conversation.item.input_audio_transcription.segment"
    
    The event type, must be conversation.item.input_audio_transcription.segment.
    - :"conversation.item.input_audio_transcription.segment"
- class McpListToolsInProgress
  
  Returned when listing MCP tools is in progress for an item.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the MCP list tools item.
  - type: :"mcp_list_tools.in_progress"
    
    The event type, must be mcp_list_tools.in_progress.
    - :"mcp_list_tools.in_progress"
- class McpListToolsCompleted
  
  Returned when listing MCP tools has completed for an item.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the MCP list tools item.
  - type: :"mcp_list_tools.completed"
    
    The event type, must be mcp_list_tools.completed.
    - :"mcp_list_tools.completed"
- class McpListToolsFailed
  
  Returned when listing MCP tools has failed for an item.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the MCP list tools item.
  - type: :"mcp_list_tools.failed"
    
    The event type, must be mcp_list_tools.failed.
    - :"mcp_list_tools.failed"
- class ResponseMcpCallArgumentsDelta
  
  Returned when MCP tool call arguments are updated during response generation.
  - delta: String
    
    The JSON-encoded arguments delta.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the MCP tool call item.
  - output_index: Integer
    
    The index of the output item in the response.
  - response_id: String
    
    The ID of the response.
  - type: :"response.mcp_call_arguments.delta"
    
    The event type, must be response.mcp_call_arguments.delta.
    - :"response.mcp_call_arguments.delta"
  - obfuscation: String
    
    If present, indicates the delta text was obfuscated.
- class ResponseMcpCallArgumentsDone
  
  Returned when MCP tool call arguments are finalized during response generation.
  - arguments: String
    
    The final JSON-encoded arguments string.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the MCP tool call item.
  - output_index: Integer
    
    The index of the output item in the response.
  - response_id: String
    
    The ID of the response.
  - type: :"response.mcp_call_arguments.done"
    
    The event type, must be response.mcp_call_arguments.done.
    - :"response.mcp_call_arguments.done"
- class ResponseMcpCallInProgress
  
  Returned when an MCP tool call has started and is in progress.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the MCP tool call item.
  - output_index: Integer
    
    The index of the output item in the response.
  - type: :"response.mcp_call.in_progress"
    
    The event type, must be response.mcp_call.in_progress.
    - :"response.mcp_call.in_progress"
- class ResponseMcpCallCompleted
  
  Returned when an MCP tool call has completed successfully.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the MCP tool call item.
  - output_index: Integer
    
    The index of the output item in the response.
  - type: :"response.mcp_call.completed"
    
    The event type, must be response.mcp_call.completed.
    - :"response.mcp_call.completed"
- class ResponseMcpCallFailed
  
  Returned when an MCP tool call has failed.
  - event_id: String
    
    The unique ID of the server event.
  - item_id: String
    
    The ID of the MCP tool call item.
  - output_index: Integer
    
    The index of the output item in the response.
  - type: :"response.mcp_call.failed"
    
    The event type, must be response.mcp_call.failed.
    - :"response.mcp_call.failed"

Realtime Session

class RealtimeSession

Realtime session object for the beta interface.
- id: String
  
  Unique identifier for the session that looks like sess_1234567890abcdef.
- expires_at: Integer
  
  Expiration timestamp for the session, in seconds since epoch.
- include: Array[:"item.input_audio_transcription.logprobs"]
  
  Additional fields to include in server outputs.
  - item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
  - :"item.input_audio_transcription.logprobs"
- input_audio_format: :pcm16 | :g711_ulaw | :g711_alaw
  
  The format of input audio. Options are pcm16, g711_ulaw, or g711_alaw. For pcm16, input audio must be 16-bit PCM at a 24kHz sample rate, single channel (mono), and little-endian byte order.
  - :pcm16
  - :g711_ulaw
  - :g711_alaw
- input_audio_noise_reduction: InputAudioNoiseReduction{ type}
  
  Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
  - type: NoiseReductionType
    
    Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
    - :near_field
    - :far_field
- input_audio_transcription: AudioTranscription
  
  Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
  - delay: :minimal | :low | :medium | 2 more
    
    Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
    - :minimal
    - :low
    - :medium
    - :high
    - :xhigh
  - language: String
    
    The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
  - model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
    
    The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
    - String = String
    - Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
      
      The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
      - :"whisper-1"
      - :"gpt-4o-mini-transcribe"
      - :"gpt-4o-mini-transcribe-2025-12-15"
      - :"gpt-4o-transcribe"
      - :"gpt-4o-transcribe-diarize"
      - :"gpt-realtime-whisper"
  - prompt: String
    
    An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
- instructions: String
  
  The default system instructions (i.e. system message) prepended to model calls. This field allows the client to guide the model on desired responses. The model can be instructed on response content and format, (e.g. "be extremely succinct", "act friendly", "here are examples of good responses") and on audio behavior (e.g. "talk quickly", "inject emotion into your voice", "laugh frequently"). The instructions are not guaranteed to be followed by the model, but they provide guidance to the model on the desired behavior.
  
  Note that the server sets default instructions which will be used if this field is not set and are visible in the session.created event at the start of the session.
- max_response_output_tokens: Integer | :inf
  
  Maximum number of output tokens for a single assistant response, inclusive of tool calls. Provide an integer between 1 and 4096 to limit output tokens, or inf for the maximum available tokens for a given model. Defaults to inf.
  - Integer = Integer
  - MaxResponseOutputTokens = :inf
    - :inf
- modalities: Array[:text | :audio]
  
  The set of modalities the model can respond with. To disable audio, set this to ["text"].
  - :text
  - :audio
- model: String | :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2025-08-28" | 13 more
  
  The Realtime model used for this session.
  - String = String
  - Model = :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2025-08-28" | 13 more
    
    The Realtime model used for this session.
    - :"gpt-realtime"
    - :"gpt-realtime-1.5"
    - :"gpt-realtime-2025-08-28"
    - :"gpt-4o-realtime-preview"
    - :"gpt-4o-realtime-preview-2024-10-01"
    - :"gpt-4o-realtime-preview-2024-12-17"
    - :"gpt-4o-realtime-preview-2025-06-03"
    - :"gpt-4o-mini-realtime-preview"
    - :"gpt-4o-mini-realtime-preview-2024-12-17"
    - :"gpt-realtime-mini"
    - :"gpt-realtime-mini-2025-10-06"
    - :"gpt-realtime-mini-2025-12-15"
    - :"gpt-audio-1.5"
    - :"gpt-audio-mini"
    - :"gpt-audio-mini-2025-10-06"
    - :"gpt-audio-mini-2025-12-15"
- object: :"realtime.session"
  
  The object type. Always realtime.session.
  - :"realtime.session"
- output_audio_format: :pcm16 | :g711_ulaw | :g711_alaw
  
  The format of output audio. Options are pcm16, g711_ulaw, or g711_alaw. For pcm16, output audio is sampled at a rate of 24kHz.
  - :pcm16
  - :g711_ulaw
  - :g711_alaw
- prompt: ResponsePrompt
  
  Reference to a prompt template and its variables. Learn more.
  - id: String
    
    The unique identifier of the prompt template to use.
  - variables: Hash[Symbol, String | ResponseInputText | ResponseInputImage | ResponseInputFile]
    
    Optional map of values to substitute in for variables in your prompt. The substitution values can either be strings, or other Response input types like images or files.
    - String = String
    - class ResponseInputText
      
      A text input to the model.
      - text: String
        
        The text input to the model.
      - type: :input_text
        
        The type of the input item. Always input_text.
        
        :input_text
    - class ResponseInputImage
      
      An image input to the model. Learn about image inputs.
      - detail: :low | :high | :auto | :original
        
        The detail level of the image to be sent to the model. One of high, low, auto, or original. Defaults to auto.
        
        :low
        
        :high
        
        :auto
        
        :original
      - type: :input_image
        
        The type of the input item. Always input_image.
        
        :input_image
      - file_id: String
        
        The ID of the file to be sent to the model.
      - image_url: String
        
        The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.
    - class ResponseInputFile
      
      A file input to the model.
      - type: :input_file
        
        The type of the input item. Always input_file.
        
        :input_file
      - detail: :low | :high
        
        The detail level of the file to be sent to the model. Use low for the default rendering behavior, or high to render the file at higher quality. Defaults to low.
        
        :low
        
        :high
      - file_data: String
        
        The content of the file to be sent to the model.
      - file_id: String
        
        The ID of the file to be sent to the model.
      - file_url: String
        
        The URL of the file to be sent to the model.
      - filename: String
        
        The name of the file to be sent to the model.
  - version: String
    
    Optional version of the prompt template.
- speed: Float
  
  The speed of the model's spoken response. 1.0 is the default speed. 0.25 is the minimum speed. 1.5 is the maximum speed. This value can only be changed in between model turns, not while a response is in progress.
- temperature: Float
  
  Sampling temperature for the model, limited to [0.6, 1.2]. For audio models a temperature of 0.8 is highly recommended for best performance.
- tool_choice: String
  
  How the model chooses tools. Options are auto, none, required, or specify a function.
- tools: Array[RealtimeFunctionTool]
  
  Tools (functions) available to the model.
  - description: String
    
    The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything).
  - name: String
    
    The name of the function.
  - parameters: untyped
    
    Parameters of the function in JSON Schema.
  - type: :function
    
    The type of the tool, i.e. function.
    - :function
- tracing: :auto | TracingConfiguration{ group_id, metadata, workflow_name}
  
  Configuration options for tracing. Set to null to disable tracing. Once tracing is enabled for a session, the configuration cannot be modified.
  
  auto will create a trace for the session with default values for the workflow name, group id, and metadata.
  - Tracing = :auto
    
    Default tracing mode for the session.
    - :auto
  - class TracingConfiguration
    
    Granular configuration for tracing.
    - group_id: String
      
      The group id to attach to this trace to enable filtering and grouping in the traces dashboard.
    - metadata: untyped
      
      The arbitrary metadata to attach to this trace to enable filtering in the traces dashboard.
    - workflow_name: String
      
      The name of the workflow to attach to this trace. This is used to name the trace in the traces dashboard.
- turn_detection: ServerVad{ type, create_response, idle_timeout_ms, 4 more} | SemanticVad{ type, create_response, eagerness, interrupt_response}
  
  Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
  
  Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
  
  Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
  
  For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
  - class ServerVad
    
    Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
    - type: :server_vad
      
      Type of turn detection, server_vad to turn on simple Server VAD.
      - :server_vad
    - create_response: bool
      
      Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
      
      If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
    - idle_timeout_ms: Integer
      
      Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
      
      The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
      
      An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
    - interrupt_response: bool
      
      Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
      
      If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
    - prefix_padding_ms: Integer
      
      Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
    - silence_duration_ms: Integer
      
      Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
    - threshold: Float
      
      Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
  - class SemanticVad
    
    Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
    - type: :semantic_vad
      
      Type of turn detection, semantic_vad to turn on Semantic VAD.
      - :semantic_vad
    - create_response: bool
      
      Whether or not to automatically generate a response when a VAD stop event occurs.
    - eagerness: :low | :medium | :high | :auto
      
      Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
      - :low
      - :medium
      - :high
      - :auto
    - interrupt_response: bool
      
      Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.
- voice: String | :alloy | :ash | :ballad | 7 more
  
  The voice the model uses to respond. Voice cannot be changed during the session once the model has responded with audio at least once. Current voice options are alloy, ash, ballad, coral, echo, sage, shimmer, and verse.
  - String = String
  - Voice = :alloy | :ash | :ballad | 7 more
    
    The voice the model uses to respond. Voice cannot be changed during the session once the model has responded with audio at least once. Current voice options are alloy, ash, ballad, coral, echo, sage, shimmer, and verse.
    - :alloy
    - :ash
    - :ballad
    - :coral
    - :echo
    - :sage
    - :shimmer
    - :verse
    - :marin
    - :cedar

Realtime Session Create Request

class RealtimeSessionCreateRequest

Realtime session object configuration.
- type: :realtime
  
  The type of session to create. Always realtime for the Realtime API.
  - :realtime
- audio: RealtimeAudioConfig
  
  Configuration for input and output audio.
  - input: RealtimeAudioConfigInput
    - format_: RealtimeAudioFormats
      
      The format of the input audio.
      - class AudioPCM
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
        
        type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
      - class AudioPCMU
        
        The G.711 μ-law format.
        
        type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
      - class AudioPCMA
        
        The G.711 A-law format.
        
        type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
    - noise_reduction: NoiseReduction{ type}
      
      Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
      - type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
    - transcription: AudioTranscription
      
      Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
      - delay: :minimal | :low | :medium | 2 more
        
        Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
        
        :minimal
        
        :low
        
        :medium
        
        :high
        
        :xhigh
      - language: String
        
        The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
      - model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        String = String
        
        Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        :"whisper-1"
        
        :"gpt-4o-mini-transcribe"
        
        :"gpt-4o-mini-transcribe-2025-12-15"
        
        :"gpt-4o-transcribe"
        
        :"gpt-4o-transcribe-diarize"
        
        :"gpt-realtime-whisper"
      - prompt: String
        
        An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
    - turn_detection: RealtimeAudioInputTurnDetection
      
      Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
      
      Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
      
      Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
      
      For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
      - class ServerVad
        
        Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
        
        type: :server_vad
        
        Type of turn detection, server_vad to turn on simple Server VAD.
        
        :server_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        idle_timeout_ms: Integer
        
        Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
        
        The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
        
        An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        prefix_padding_ms: Integer
        
        Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
        
        silence_duration_ms: Integer
        
        Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
        
        threshold: Float
        
        Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
      - class SemanticVad
        
        Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
        
        type: :semantic_vad
        
        Type of turn detection, semantic_vad to turn on Semantic VAD.
        
        :semantic_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs.
        
        eagerness: :low | :medium | :high | :auto
        
        Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
        
        :low
        
        :medium
        
        :high
        
        :auto
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.
  - output: RealtimeAudioConfigOutput
    - format_: RealtimeAudioFormats
      
      The format of the output audio.
    - speed: Float
      
      The speed of the model's spoken response as a multiple of the original speed. 1.0 is the default speed. 0.25 is the minimum speed. 1.5 is the maximum speed. This value can only be changed in between model turns, not while a response is in progress.
      
      This parameter is a post-processing adjustment to the audio after it is generated, it's also possible to prompt the model to speak faster or slower.
    - voice: String | :alloy | :ash | :ballad | 7 more | ID{ id}
      
      The voice the model uses to respond. Supported built-in voices are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. You may also provide a custom voice object with an id, for example { "id": "voice_1234" }. Voice cannot be changed during the session once the model has responded with audio at least once. We recommend marin and cedar for best quality.
      - String = String
      - Voice = :alloy | :ash | :ballad | 7 more
        
        :alloy
        
        :ash
        
        :ballad
        
        :coral
        
        :echo
        
        :sage
        
        :shimmer
        
        :verse
        
        :marin
        
        :cedar
      - class ID
        
        Custom voice reference.
        
        id: String
        
        The custom voice ID, e.g. voice_1234.
- include: Array[:"item.input_audio_transcription.logprobs"]
  
  Additional fields to include in server outputs.
  
  item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
  - :"item.input_audio_transcription.logprobs"
- instructions: String
  
  The default system instructions (i.e. system message) prepended to model calls. This field allows the client to guide the model on desired responses. The model can be instructed on response content and format, (e.g. "be extremely succinct", "act friendly", "here are examples of good responses") and on audio behavior (e.g. "talk quickly", "inject emotion into your voice", "laugh frequently"). The instructions are not guaranteed to be followed by the model, but they provide guidance to the model on the desired behavior.
  
  Note that the server sets default instructions which will be used if this field is not set and are visible in the session.created event at the start of the session.
- max_output_tokens: Integer | :inf
  
  Maximum number of output tokens for a single assistant response, inclusive of tool calls. Provide an integer between 1 and 4096 to limit output tokens, or inf for the maximum available tokens for a given model. Defaults to inf.
  - Integer = Integer
  - MaxOutputTokens = :inf
    - :inf
- model: String | :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
  
  The Realtime model used for this session.
  - String = String
  - Model = :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
    
    The Realtime model used for this session.
    - :"gpt-realtime"
    - :"gpt-realtime-1.5"
    - :"gpt-realtime-2"
    - :"gpt-realtime-2025-08-28"
    - :"gpt-4o-realtime-preview"
    - :"gpt-4o-realtime-preview-2024-10-01"
    - :"gpt-4o-realtime-preview-2024-12-17"
    - :"gpt-4o-realtime-preview-2025-06-03"
    - :"gpt-4o-mini-realtime-preview"
    - :"gpt-4o-mini-realtime-preview-2024-12-17"
    - :"gpt-realtime-mini"
    - :"gpt-realtime-mini-2025-10-06"
    - :"gpt-realtime-mini-2025-12-15"
    - :"gpt-audio-1.5"
    - :"gpt-audio-mini"
    - :"gpt-audio-mini-2025-10-06"
    - :"gpt-audio-mini-2025-12-15"
- output_modalities: Array[:text | :audio]
  
  The set of modalities the model can respond with. It defaults to ["audio"], indicating that the model will respond with audio plus a transcript. ["text"] can be used to make the model respond with text only. It is not possible to request both text and audio at the same time.
  - :text
  - :audio
- parallel_tool_calls: bool
  
  Whether the model may call multiple tools in parallel. Only supported by reasoning Realtime models such as gpt-realtime-2.
- prompt: ResponsePrompt
  
  Reference to a prompt template and its variables. Learn more.
  - id: String
    
    The unique identifier of the prompt template to use.
  - variables: Hash[Symbol, String | ResponseInputText | ResponseInputImage | ResponseInputFile]
    
    Optional map of values to substitute in for variables in your prompt. The substitution values can either be strings, or other Response input types like images or files.
    - String = String
    - class ResponseInputText
      
      A text input to the model.
      - text: String
        
        The text input to the model.
      - type: :input_text
        
        The type of the input item. Always input_text.
        
        :input_text
    - class ResponseInputImage
      
      An image input to the model. Learn about image inputs.
      - detail: :low | :high | :auto | :original
        
        The detail level of the image to be sent to the model. One of high, low, auto, or original. Defaults to auto.
        
        :low
        
        :high
        
        :auto
        
        :original
      - type: :input_image
        
        The type of the input item. Always input_image.
        
        :input_image
      - file_id: String
        
        The ID of the file to be sent to the model.
      - image_url: String
        
        The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.
    - class ResponseInputFile
      
      A file input to the model.
      - type: :input_file
        
        The type of the input item. Always input_file.
        
        :input_file
      - detail: :low | :high
        
        The detail level of the file to be sent to the model. Use low for the default rendering behavior, or high to render the file at higher quality. Defaults to low.
        
        :low
        
        :high
      - file_data: String
        
        The content of the file to be sent to the model.
      - file_id: String
        
        The ID of the file to be sent to the model.
      - file_url: String
        
        The URL of the file to be sent to the model.
      - filename: String
        
        The name of the file to be sent to the model.
  - version: String
    
    Optional version of the prompt template.
- reasoning: RealtimeReasoning
  
  Configuration for reasoning-capable Realtime models such as gpt-realtime-2.
  - effort: RealtimeReasoningEffort
    
    Constrains effort on reasoning for reasoning-capable Realtime models such as gpt-realtime-2.
    - :minimal
    - :low
    - :medium
    - :high
    - :xhigh
- tool_choice: RealtimeToolChoiceConfig
  
  How the model chooses tools. Provide one of the string modes or force a specific function/MCP tool.
  - ToolChoiceOptions = :none | :auto | :required
    
    Controls which (if any) tool is called by the model.
    
    none means the model will not call any tool and instead generates a message.
    
    auto means the model can pick between generating a message or calling one or more tools.
    
    required means the model must call one or more tools.
    - :none
    - :auto
    - :required
  - class ToolChoiceFunction
    
    Use this option to force the model to call a specific function.
    - name: String
      
      The name of the function to call.
    - type: :function
      
      For function calling, the type is always function.
      - :function
  - class ToolChoiceMcp
    
    Use this option to force the model to call a specific tool on a remote MCP server.
    - server_label: String
      
      The label of the MCP server to use.
    - type: :mcp
      
      For MCP tools, the type is always mcp.
      - :mcp
    - name: String
      
      The name of the tool to call on the server.
- tools: RealtimeToolsConfig
  
  Tools available to the model.
  - class RealtimeFunctionTool
    - description: String
      
      The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything).
    - name: String
      
      The name of the function.
    - parameters: untyped
      
      Parameters of the function in JSON Schema.
    - type: :function
      
      The type of the tool, i.e. function.
      - :function
  - class Mcp
    
    Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.
    - server_label: String
      
      A label for this MCP server, used to identify it in tool calls.
    - type: :mcp
      
      The type of the MCP tool. Always mcp.
      - :mcp
    - allowed_tools: Array[String] | McpToolFilter{ read_only, tool_names}
      
      List of allowed tool names or a filter object.
      - McpAllowedTools = Array[String]
        
        A string array of allowed tool names
      - class McpToolFilter
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
    - authorization: String
      
      An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.
    - connector_id: :connector_dropbox | :connector_gmail | :connector_googlecalendar | 5 more
      
      Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here.
      
      Currently supported connector_id values are:
      - Dropbox: connector_dropbox
      - Gmail: connector_gmail
      - Google Calendar: connector_googlecalendar
      - Google Drive: connector_googledrive
      - Microsoft Teams: connector_microsoftteams
      - Outlook Calendar: connector_outlookcalendar
      - Outlook Email: connector_outlookemail
      - SharePoint: connector_sharepoint
      - :connector_dropbox
      - :connector_gmail
      - :connector_googlecalendar
      - :connector_googledrive
      - :connector_microsoftteams
      - :connector_outlookcalendar
      - :connector_outlookemail
      - :connector_sharepoint
    - defer_loading: bool
      
      Whether this MCP tool is deferred and discovered via tool search.
    - headers: Hash[Symbol, String]
      
      Optional HTTP headers to send to the MCP server. Use for authentication or other purposes.
    - require_approval: McpToolApprovalFilter{ always, never} | :always | :never
      
      Specify which of the MCP server's tools require approval.
      - class McpToolApprovalFilter
        
        Specify which of the MCP server's tools require approval. Can be always, never, or a filter object associated with tools that require approval.
        
        always: Always{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        never: Never{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
      - McpToolApprovalSetting = :always | :never
        
        Specify a single approval policy for all tools. One of always or never. When set to always, all tools will require approval. When set to never, all tools will not require approval.
        
        :always
        
        :never
    - server_description: String
      
      Optional description of the MCP server, used to provide more context.
    - server_url: String
      
      The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided.
    - tunnel_id: String
      
      The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided.
- tracing: RealtimeTracingConfig
  
  Realtime API can write session traces to the Traces Dashboard. Set to null to disable tracing. Once tracing is enabled for a session, the configuration cannot be modified.
  
  auto will create a trace for the session with default values for the workflow name, group id, and metadata.
  - RealtimeTracingConfig = :auto
    
    Enables tracing and sets default values for tracing configuration options. Always auto.
    - :auto
  - class TracingConfiguration
    
    Granular configuration for tracing.
    - group_id: String
      
      The group id to attach to this trace to enable filtering and grouping in the Traces Dashboard.
    - metadata: untyped
      
      The arbitrary metadata to attach to this trace to enable filtering in the Traces Dashboard.
    - workflow_name: String
      
      The name of the workflow to attach to this trace. This is used to name the trace in the Traces Dashboard.
- truncation: RealtimeTruncation
  
  When the number of tokens in a conversation exceeds the model's input token limit, the conversation be truncated, meaning messages (starting from the oldest) will not be included in the model's context. A 32k context model with 4,096 max output tokens can only include 28,224 tokens in the context before truncation occurs.
  
  Clients can configure truncation behavior to truncate with a lower max token limit, which is an effective way to control token usage and cost.
  
  Truncation will reduce the number of cached tokens on the next turn (busting the cache), since messages are dropped from the beginning of the context. However, clients can also configure truncation to retain messages up to a fraction of the maximum context size, which will reduce the need for future truncations and thus improve the cache rate.
  
  Truncation can be disabled entirely, which means the server will never truncate but would instead return an error if the conversation exceeds the model's input token limit.
  - RealtimeTruncationStrategy = :auto | :disabled
    
    The truncation strategy to use for the session. auto is the default truncation strategy. disabled will disable truncation and emit errors when the conversation exceeds the input token limit.
    - :auto
    - :disabled
  - class RealtimeTruncationRetentionRatio
    
    Retain a fraction of the conversation tokens when the conversation exceeds the input token limit. This allows you to amortize truncations across multiple turns, which can help improve cached token usage.
    - retention_ratio: Float
      
      Fraction of post-instruction conversation tokens to retain (0.0 - 1.0) when the conversation exceeds the input token limit. Setting this to 0.8 means that messages will be dropped until 80% of the maximum allowed tokens are used. This helps reduce the frequency of truncations and improve cache rates.
    - type: :retention_ratio
      
      Use retention ratio truncation.
      - :retention_ratio
    - token_limits: TokenLimits{ post_instructions}
      
      Optional custom token limits for this truncation strategy. If not provided, the model's default token limits will be used.
      - post_instructions: Integer
        
        Maximum tokens allowed in the conversation after instructions (which including tool definitions). For example, setting this to 5,000 would mean that truncation would occur when the conversation exceeds 5,000 tokens after instructions. This cannot be higher than the model's context window size minus the maximum output tokens.

Realtime Tool Choice Config

RealtimeToolChoiceConfig = ToolChoiceOptions | ToolChoiceFunction | ToolChoiceMcp

How the model chooses tools. Provide one of the string modes or force a specific function/MCP tool.
- ToolChoiceOptions = :none | :auto | :required
  
  Controls which (if any) tool is called by the model.
  
  none means the model will not call any tool and instead generates a message.
  
  auto means the model can pick between generating a message or calling one or more tools.
  
  required means the model must call one or more tools.
  - :none
  - :auto
  - :required
- class ToolChoiceFunction
  
  Use this option to force the model to call a specific function.
  - name: String
    
    The name of the function to call.
  - type: :function
    
    For function calling, the type is always function.
    - :function
- class ToolChoiceMcp
  
  Use this option to force the model to call a specific tool on a remote MCP server.
  - server_label: String
    
    The label of the MCP server to use.
  - type: :mcp
    
    For MCP tools, the type is always mcp.
    - :mcp
  - name: String
    
    The name of the tool to call on the server.

Realtime Tools Config

RealtimeToolsConfig = Array[RealtimeToolsConfigUnion]

Tools available to the model.
- class RealtimeFunctionTool
  - description: String
    
    The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything).
  - name: String
    
    The name of the function.
  - parameters: untyped
    
    Parameters of the function in JSON Schema.
  - type: :function
    
    The type of the tool, i.e. function.
    - :function
- class Mcp
  
  Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.
  - server_label: String
    
    A label for this MCP server, used to identify it in tool calls.
  - type: :mcp
    
    The type of the MCP tool. Always mcp.
    - :mcp
  - allowed_tools: Array[String] | McpToolFilter{ read_only, tool_names}
    
    List of allowed tool names or a filter object.
    - McpAllowedTools = Array[String]
      
      A string array of allowed tool names
    - class McpToolFilter
      
      A filter object to specify which tools are allowed.
      - read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
      - tool_names: Array[String]
        
        List of allowed tool names.
  - authorization: String
    
    An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.
  - connector_id: :connector_dropbox | :connector_gmail | :connector_googlecalendar | 5 more
    
    Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here.
    
    Currently supported connector_id values are:
    - Dropbox: connector_dropbox
    - Gmail: connector_gmail
    - Google Calendar: connector_googlecalendar
    - Google Drive: connector_googledrive
    - Microsoft Teams: connector_microsoftteams
    - Outlook Calendar: connector_outlookcalendar
    - Outlook Email: connector_outlookemail
    - SharePoint: connector_sharepoint
    - :connector_dropbox
    - :connector_gmail
    - :connector_googlecalendar
    - :connector_googledrive
    - :connector_microsoftteams
    - :connector_outlookcalendar
    - :connector_outlookemail
    - :connector_sharepoint
  - defer_loading: bool
    
    Whether this MCP tool is deferred and discovered via tool search.
  - headers: Hash[Symbol, String]
    
    Optional HTTP headers to send to the MCP server. Use for authentication or other purposes.
  - require_approval: McpToolApprovalFilter{ always, never} | :always | :never
    
    Specify which of the MCP server's tools require approval.
    - class McpToolApprovalFilter
      
      Specify which of the MCP server's tools require approval. Can be always, never, or a filter object associated with tools that require approval.
      - always: Always{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
      - never: Never{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
    - McpToolApprovalSetting = :always | :never
      
      Specify a single approval policy for all tools. One of always or never. When set to always, all tools will require approval. When set to never, all tools will not require approval.
      - :always
      - :never
  - server_description: String
    
    Optional description of the MCP server, used to provide more context.
  - server_url: String
    
    The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided.
  - tunnel_id: String
    
    The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided.

Realtime Tools Config Union

RealtimeToolsConfigUnion = RealtimeFunctionTool | Mcp{ server_label, type, allowed_tools, 8 more}

Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.
- class RealtimeFunctionTool
  - description: String
    
    The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything).
  - name: String
    
    The name of the function.
  - parameters: untyped
    
    Parameters of the function in JSON Schema.
  - type: :function
    
    The type of the tool, i.e. function.
    - :function
- class Mcp
  
  Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.
  - server_label: String
    
    A label for this MCP server, used to identify it in tool calls.
  - type: :mcp
    
    The type of the MCP tool. Always mcp.
    - :mcp
  - allowed_tools: Array[String] | McpToolFilter{ read_only, tool_names}
    
    List of allowed tool names or a filter object.
    - McpAllowedTools = Array[String]
      
      A string array of allowed tool names
    - class McpToolFilter
      
      A filter object to specify which tools are allowed.
      - read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
      - tool_names: Array[String]
        
        List of allowed tool names.
  - authorization: String
    
    An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.
  - connector_id: :connector_dropbox | :connector_gmail | :connector_googlecalendar | 5 more
    
    Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here.
    
    Currently supported connector_id values are:
    - Dropbox: connector_dropbox
    - Gmail: connector_gmail
    - Google Calendar: connector_googlecalendar
    - Google Drive: connector_googledrive
    - Microsoft Teams: connector_microsoftteams
    - Outlook Calendar: connector_outlookcalendar
    - Outlook Email: connector_outlookemail
    - SharePoint: connector_sharepoint
    - :connector_dropbox
    - :connector_gmail
    - :connector_googlecalendar
    - :connector_googledrive
    - :connector_microsoftteams
    - :connector_outlookcalendar
    - :connector_outlookemail
    - :connector_sharepoint
  - defer_loading: bool
    
    Whether this MCP tool is deferred and discovered via tool search.
  - headers: Hash[Symbol, String]
    
    Optional HTTP headers to send to the MCP server. Use for authentication or other purposes.
  - require_approval: McpToolApprovalFilter{ always, never} | :always | :never
    
    Specify which of the MCP server's tools require approval.
    - class McpToolApprovalFilter
      
      Specify which of the MCP server's tools require approval. Can be always, never, or a filter object associated with tools that require approval.
      - always: Always{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
      - never: Never{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
    - McpToolApprovalSetting = :always | :never
      
      Specify a single approval policy for all tools. One of always or never. When set to always, all tools will require approval. When set to never, all tools will not require approval.
      - :always
      - :never
  - server_description: String
    
    Optional description of the MCP server, used to provide more context.
  - server_url: String
    
    The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided.
  - tunnel_id: String
    
    The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided.

Realtime Tracing Config

RealtimeTracingConfig = :auto | TracingConfiguration{ group_id, metadata, workflow_name}

Realtime API can write session traces to the Traces Dashboard. Set to null to disable tracing. Once tracing is enabled for a session, the configuration cannot be modified.

auto will create a trace for the session with default values for the workflow name, group id, and metadata.
- RealtimeTracingConfig = :auto
  
  Enables tracing and sets default values for tracing configuration options. Always auto.
  - :auto
- class TracingConfiguration
  
  Granular configuration for tracing.
  - group_id: String
    
    The group id to attach to this trace to enable filtering and grouping in the Traces Dashboard.
  - metadata: untyped
    
    The arbitrary metadata to attach to this trace to enable filtering in the Traces Dashboard.
  - workflow_name: String
    
    The name of the workflow to attach to this trace. This is used to name the trace in the Traces Dashboard.

Realtime Transcription Session Audio

class RealtimeTranscriptionSessionAudio

Configuration for input and output audio.
- input: RealtimeTranscriptionSessionAudioInput
  - format_: RealtimeAudioFormats
    
    The PCM audio format. Only a 24kHz sample rate is supported.
    - class AudioPCM
      
      The PCM audio format. Only a 24kHz sample rate is supported.
      - rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
      - type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
    - class AudioPCMU
      
      The G.711 μ-law format.
      - type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
    - class AudioPCMA
      
      The G.711 A-law format.
      - type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
  - noise_reduction: NoiseReduction{ type}
    
    Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
    - type: NoiseReductionType
      
      Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
      - :near_field
      - :far_field
  - transcription: AudioTranscription
    
    Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
    - delay: :minimal | :low | :medium | 2 more
      
      Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
      - :minimal
      - :low
      - :medium
      - :high
      - :xhigh
    - language: String
      
      The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
    - model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
      
      The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
      - String = String
      - Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        :"whisper-1"
        
        :"gpt-4o-mini-transcribe"
        
        :"gpt-4o-mini-transcribe-2025-12-15"
        
        :"gpt-4o-transcribe"
        
        :"gpt-4o-transcribe-diarize"
        
        :"gpt-realtime-whisper"
    - prompt: String
      
      An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
  - turn_detection: RealtimeTranscriptionSessionAudioInputTurnDetection
    
    Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
    
    Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
    
    Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
    
    For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
    - class ServerVad
      
      Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
      - type: :server_vad
        
        Type of turn detection, server_vad to turn on simple Server VAD.
        
        :server_vad
      - create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
      - idle_timeout_ms: Integer
        
        Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
        
        The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
        
        An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
      - interrupt_response: bool
        
        Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
      - prefix_padding_ms: Integer
        
        Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
      - silence_duration_ms: Integer
        
        Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
      - threshold: Float
        
        Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
    - class SemanticVad
      
      Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
      - type: :semantic_vad
        
        Type of turn detection, semantic_vad to turn on Semantic VAD.
        
        :semantic_vad
      - create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs.
      - eagerness: :low | :medium | :high | :auto
        
        Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
        
        :low
        
        :medium
        
        :high
        
        :auto
      - interrupt_response: bool
        
        Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.

Realtime Transcription Session Audio Input

class RealtimeTranscriptionSessionAudioInput
- format_: RealtimeAudioFormats
  
  The PCM audio format. Only a 24kHz sample rate is supported.
  - class AudioPCM
    
    The PCM audio format. Only a 24kHz sample rate is supported.
    - rate: 24000
      
      The sample rate of the audio. Always 24000.
      - 24000
    - type: :"audio/pcm"
      
      The audio format. Always audio/pcm.
      - :"audio/pcm"
  - class AudioPCMU
    
    The G.711 μ-law format.
    - type: :"audio/pcmu"
      
      The audio format. Always audio/pcmu.
      - :"audio/pcmu"
  - class AudioPCMA
    
    The G.711 A-law format.
    - type: :"audio/pcma"
      
      The audio format. Always audio/pcma.
      - :"audio/pcma"
- noise_reduction: NoiseReduction{ type}
  
  Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
  - type: NoiseReductionType
    
    Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
    - :near_field
    - :far_field
- transcription: AudioTranscription
  
  Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
  - delay: :minimal | :low | :medium | 2 more
    
    Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
    - :minimal
    - :low
    - :medium
    - :high
    - :xhigh
  - language: String
    
    The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
  - model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
    
    The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
    - String = String
    - Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
      
      The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
      - :"whisper-1"
      - :"gpt-4o-mini-transcribe"
      - :"gpt-4o-mini-transcribe-2025-12-15"
      - :"gpt-4o-transcribe"
      - :"gpt-4o-transcribe-diarize"
      - :"gpt-realtime-whisper"
  - prompt: String
    
    An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
- turn_detection: RealtimeTranscriptionSessionAudioInputTurnDetection
  
  Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
  
  Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
  
  Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
  
  For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
  - class ServerVad
    
    Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
    - type: :server_vad
      
      Type of turn detection, server_vad to turn on simple Server VAD.
      - :server_vad
    - create_response: bool
      
      Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
      
      If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
    - idle_timeout_ms: Integer
      
      Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
      
      The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
      
      An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
    - interrupt_response: bool
      
      Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
      
      If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
    - prefix_padding_ms: Integer
      
      Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
    - silence_duration_ms: Integer
      
      Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
    - threshold: Float
      
      Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
  - class SemanticVad
    
    Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
    - type: :semantic_vad
      
      Type of turn detection, semantic_vad to turn on Semantic VAD.
      - :semantic_vad
    - create_response: bool
      
      Whether or not to automatically generate a response when a VAD stop event occurs.
    - eagerness: :low | :medium | :high | :auto
      
      Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
      - :low
      - :medium
      - :high
      - :auto
    - interrupt_response: bool
      
      Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.

Realtime Transcription Session Audio Input Turn Detection

RealtimeTranscriptionSessionAudioInputTurnDetection = ServerVad{ type, create_response, idle_timeout_ms, 4 more} | SemanticVad{ type, create_response, eagerness, interrupt_response}

Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.

Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.

Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.

For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
- class ServerVad
  
  Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
  - type: :server_vad
    
    Type of turn detection, server_vad to turn on simple Server VAD.
    - :server_vad
  - create_response: bool
    
    Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
    
    If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
  - idle_timeout_ms: Integer
    
    Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
    
    The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
    
    An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
  - interrupt_response: bool
    
    Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
    
    If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
  - prefix_padding_ms: Integer
    
    Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
  - silence_duration_ms: Integer
    
    Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
  - threshold: Float
    
    Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
- class SemanticVad
  
  Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
  - type: :semantic_vad
    
    Type of turn detection, semantic_vad to turn on Semantic VAD.
    - :semantic_vad
  - create_response: bool
    
    Whether or not to automatically generate a response when a VAD stop event occurs.
  - eagerness: :low | :medium | :high | :auto
    
    Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
    - :low
    - :medium
    - :high
    - :auto
  - interrupt_response: bool
    
    Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.

Realtime Transcription Session Create Request

class RealtimeTranscriptionSessionCreateRequest

Realtime transcription session object configuration.
- type: :transcription
  
  The type of session to create. Always transcription for transcription sessions.
  - :transcription
- audio: RealtimeTranscriptionSessionAudio
  
  Configuration for input and output audio.
  - input: RealtimeTranscriptionSessionAudioInput
    - format_: RealtimeAudioFormats
      
      The PCM audio format. Only a 24kHz sample rate is supported.
      - class AudioPCM
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
        
        type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
      - class AudioPCMU
        
        The G.711 μ-law format.
        
        type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
      - class AudioPCMA
        
        The G.711 A-law format.
        
        type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
    - noise_reduction: NoiseReduction{ type}
      
      Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
      - type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
    - transcription: AudioTranscription
      
      Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
      - delay: :minimal | :low | :medium | 2 more
        
        Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
        
        :minimal
        
        :low
        
        :medium
        
        :high
        
        :xhigh
      - language: String
        
        The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
      - model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        String = String
        
        Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        :"whisper-1"
        
        :"gpt-4o-mini-transcribe"
        
        :"gpt-4o-mini-transcribe-2025-12-15"
        
        :"gpt-4o-transcribe"
        
        :"gpt-4o-transcribe-diarize"
        
        :"gpt-realtime-whisper"
      - prompt: String
        
        An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
    - turn_detection: RealtimeTranscriptionSessionAudioInputTurnDetection
      
      Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
      
      Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
      
      Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
      
      For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
      - class ServerVad
        
        Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
        
        type: :server_vad
        
        Type of turn detection, server_vad to turn on simple Server VAD.
        
        :server_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        idle_timeout_ms: Integer
        
        Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
        
        The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
        
        An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        prefix_padding_ms: Integer
        
        Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
        
        silence_duration_ms: Integer
        
        Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
        
        threshold: Float
        
        Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
      - class SemanticVad
        
        Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
        
        type: :semantic_vad
        
        Type of turn detection, semantic_vad to turn on Semantic VAD.
        
        :semantic_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs.
        
        eagerness: :low | :medium | :high | :auto
        
        Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
        
        :low
        
        :medium
        
        :high
        
        :auto
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.
- include: Array[:"item.input_audio_transcription.logprobs"]
  
  Additional fields to include in server outputs.
  
  item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
  - :"item.input_audio_transcription.logprobs"

Realtime Translation Client Event

RealtimeTranslationClientEvent = RealtimeTranslationSessionUpdateEvent | RealtimeTranslationInputAudioBufferAppendEvent | RealtimeTranslationSessionCloseEvent

A Realtime translation client event.
- class RealtimeTranslationSessionUpdateEvent
  
  Send this event to update the translation session configuration. Translation sessions support updates to audio.output.language, audio.input.transcription, and audio.input.noise_reduction.
  - session: RealtimeTranslationSessionUpdateRequest
    
    Translation session fields to update. The session type and model are set at creation and cannot be changed with session.update.
    - audio: Audio{ input, output}
      
      Configuration for translation input and output audio.
      - input: Input{ noise_reduction, transcription}
        
        noise_reduction: NoiseReduction{ type}
        
        Optional input noise reduction. Set to null to disable it.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
        
        transcription: Transcription{ model}
        
        Optional source-language transcription. When configured, the server emits session.input_transcript.delta events. Translation itself still runs from the input audio stream.
        
        model: String
        
        The transcription model to use for source transcript deltas.
      - output: Output{ language}
        
        language: String
        
        Target language for translated output audio and transcript deltas.
  - type: :"session.update"
    
    The event type, must be session.update.
    - :"session.update"
  - event_id: String
    
    Optional client-generated ID used to identify this event.
- class RealtimeTranslationInputAudioBufferAppendEvent
  
  Send this event to append audio bytes to the translation session input audio buffer.
  
  WebSocket translation sessions accept base64-encoded 24 kHz PCM16 mono little-endian raw audio bytes. Unsupported websocket audio formats return a validation error because lower-quality audio materially degrades translation quality.
  
  Translation consumes 200 ms engine frames. For best realtime behavior, append audio in 200 ms chunks. If a chunk is shorter, the server buffers it until it has enough audio for one frame. If a chunk is longer, the server splits it into 200 ms frames and enqueues them back-to-back.
  
  Keep appending silence while the session is active. If a client stops sending audio and later resumes, model time treats the resumed audio as contiguous with the previous audio rather than as a real-world pause.
  - audio: String
    
    Base64-encoded 24 kHz PCM16 mono audio bytes.
  - type: :"session.input_audio_buffer.append"
    
    The event type, must be session.input_audio_buffer.append.
    - :"session.input_audio_buffer.append"
  - event_id: String
    
    Optional client-generated ID used to identify this event.
- class RealtimeTranslationSessionCloseEvent
  
  Gracefully close the realtime translation session. The server flushes pending input audio and emits any remaining translated output before closing the session.
  - type: :"session.close"
    
    The event type, must be session.close.
    - :"session.close"
  - event_id: String
    
    Optional client-generated ID used to identify this event.

Realtime Translation Client Secret Create Request

class RealtimeTranslationClientSecretCreateRequest

Create a translation session and client secret for the Realtime API.
- session: RealtimeTranslationSessionCreateRequest
  
  Realtime translation session configuration. Translation sessions stream source audio in and translated audio plus transcript deltas out continuously.
  - model: String
    
    The Realtime translation model used for this session.
  - audio: Audio{ input, output}
    
    Configuration for translation input and output audio.
    - input: Input{ noise_reduction, transcription}
      - noise_reduction: NoiseReduction{ type}
        
        Optional input noise reduction. Set to null to disable it.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
      - transcription: Transcription{ model}
        
        Optional source-language transcription. When configured, the server emits session.input_transcript.delta events. Translation itself still runs from the input audio stream.
        
        model: String
        
        The transcription model to use for source transcript deltas.
    - output: Output{ language}
      - language: String
        
        Target language for translated output audio and transcript deltas.
- expires_after: ExpiresAfter{ anchor, seconds}
  
  Configuration for the client secret expiration. Expiration refers to the time after which a client secret will no longer be valid for creating sessions. The session itself may continue after that time once started. A secret can be used to create multiple sessions until it expires.
  - anchor: :created_at
    
    The anchor point for the client secret expiration, meaning that seconds will be added to the created_at time of the client secret to produce an expiration timestamp. Only created_at is currently supported.
    - :created_at
  - seconds: Integer
    
    The number of seconds from the anchor point to the expiration. Select a value between 10 and 7200 (2 hours). This default to 600 seconds (10 minutes) if not specified.

Realtime Translation Client Secret Create Response

class RealtimeTranslationClientSecretCreateResponse

Response from creating a translation session and client secret for the Realtime API.
- expires_at: Integer
  
  Expiration timestamp for the client secret, in seconds since epoch.
- session: RealtimeTranslationSession
  
  A Realtime translation session. Translation sessions continuously translate input audio into the configured output language.
  - id: String
    
    Unique identifier for the session that looks like sess_1234567890abcdef.
  - audio: Audio{ input, output}
    
    Configuration for translation input and output audio.
    - input: Input{ noise_reduction, transcription}
      - noise_reduction: NoiseReduction{ type}
        
        Optional input noise reduction.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
      - transcription: Transcription{ model}
        
        Optional source-language transcription. When configured, the server emits session.input_transcript.delta events. Translation itself still runs from the input audio stream.
        
        model: String
        
        The transcription model used for source transcript deltas.
    - output: Output{ language}
      - language: String
        
        Target language for translated output audio and transcript deltas.
  - expires_at: Integer
    
    Expiration timestamp for the session, in seconds since epoch.
  - model: String
    
    The Realtime translation model used for this session. This field is set at session creation and cannot be changed with session.update.
  - type: :translation
    
    The session type. Always translation for Realtime translation sessions.
    - :translation
- value: String
  
  The generated client secret value.

Realtime Translation Input Audio Buffer Append Event

class RealtimeTranslationInputAudioBufferAppendEvent

Send this event to append audio bytes to the translation session input audio buffer.

WebSocket translation sessions accept base64-encoded 24 kHz PCM16 mono little-endian raw audio bytes. Unsupported websocket audio formats return a validation error because lower-quality audio materially degrades translation quality.

Translation consumes 200 ms engine frames. For best realtime behavior, append audio in 200 ms chunks. If a chunk is shorter, the server buffers it until it has enough audio for one frame. If a chunk is longer, the server splits it into 200 ms frames and enqueues them back-to-back.

Keep appending silence while the session is active. If a client stops sending audio and later resumes, model time treats the resumed audio as contiguous with the previous audio rather than as a real-world pause.
- audio: String
  
  Base64-encoded 24 kHz PCM16 mono audio bytes.
- type: :"session.input_audio_buffer.append"
  
  The event type, must be session.input_audio_buffer.append.
  - :"session.input_audio_buffer.append"
- event_id: String
  
  Optional client-generated ID used to identify this event.

Realtime Translation Input Transcript Delta Event

class RealtimeTranslationInputTranscriptDeltaEvent

Returned when optional source-language transcript text is available. This event is emitted only when audio.input.transcription is configured.

Transcript deltas are append-only text fragments. Clients should not insert unconditional spaces between deltas.
- delta: String
  
  Append-only source-language transcript text.
- event_id: String
  
  The unique ID of the server event.
- type: :"session.input_transcript.delta"
  
  The event type, must be session.input_transcript.delta.
  - :"session.input_transcript.delta"
- elapsed_ms: Integer
  
  Timing metadata for stream alignment, derived from the translation frame when available. It advances in 200 ms increments, but multiple transcript deltas may share the same elapsed_ms. Treat it as alignment metadata, not a unique transcript-delta identifier.

Realtime Translation Output Audio Delta Event

class RealtimeTranslationOutputAudioDeltaEvent

Returned when translated output audio is available. Output audio deltas are 200 ms frames of PCM16 audio.
- delta: String
  
  Base64-encoded translated audio data.
- event_id: String
  
  The unique ID of the server event.
- type: :"session.output_audio.delta"
  
  The event type, must be session.output_audio.delta.
  - :"session.output_audio.delta"
- channels: Integer
  
  Number of audio channels.
- elapsed_ms: Integer
  
  Timing metadata for stream alignment, derived from the translation frame when available. Treat elapsed_ms as alignment metadata, not a unique event identifier.
- format_: :pcm16
  
  Audio encoding for delta.
  - :pcm16
- sample_rate: Integer
  
  Sample rate of the audio delta.

Realtime Translation Output Transcript Delta Event

class RealtimeTranslationOutputTranscriptDeltaEvent

Returned when translated transcript text is available.

Transcript deltas are append-only text fragments. Clients should not insert unconditional spaces between deltas.
- delta: String
  
  Append-only transcript text for the translated output audio.
- event_id: String
  
  The unique ID of the server event.
- type: :"session.output_transcript.delta"
  
  The event type, must be session.output_transcript.delta.
  - :"session.output_transcript.delta"
- elapsed_ms: Integer
  
  Timing metadata for stream alignment, derived from the translation frame when available. It advances in 200 ms increments, but multiple transcript deltas may share the same elapsed_ms. Treat it as alignment metadata, not a unique transcript-delta identifier.

Realtime Translation Server Event

RealtimeTranslationServerEvent = RealtimeErrorEvent | RealtimeTranslationSessionCreatedEvent | RealtimeTranslationSessionUpdatedEvent | 4 more

A Realtime translation server event.
- class RealtimeErrorEvent
  
  Returned when an error occurs, which could be a client problem or a server problem. Most errors are recoverable and the session will stay open, we recommend to implementors to monitor and log error messages by default.
  - error: RealtimeError
    
    Details of the error.
    - message: String
      
      A human-readable error message.
    - type: String
      
      The type of error (e.g., "invalid_request_error", "server_error").
    - code: String
      
      Error code, if any.
    - event_id: String
      
      The event_id of the client event that caused the error, if applicable.
    - param: String
      
      Parameter related to the error, if any.
  - event_id: String
    
    The unique ID of the server event.
  - type: :error
    
    The event type, must be error.
    - :error
- class RealtimeTranslationSessionCreatedEvent
  
  Returned when a translation session is created. Emitted automatically when a new connection is established as the first server event. This event contains the default translation session configuration.
  - event_id: String
    
    The unique ID of the server event.
  - session: RealtimeTranslationSession
    
    The translation session configuration.
    - id: String
      
      Unique identifier for the session that looks like sess_1234567890abcdef.
    - audio: Audio{ input, output}
      
      Configuration for translation input and output audio.
      - input: Input{ noise_reduction, transcription}
        
        noise_reduction: NoiseReduction{ type}
        
        Optional input noise reduction.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
        
        transcription: Transcription{ model}
        
        Optional source-language transcription. When configured, the server emits session.input_transcript.delta events. Translation itself still runs from the input audio stream.
        
        model: String
        
        The transcription model used for source transcript deltas.
      - output: Output{ language}
        
        language: String
        
        Target language for translated output audio and transcript deltas.
    - expires_at: Integer
      
      Expiration timestamp for the session, in seconds since epoch.
    - model: String
      
      The Realtime translation model used for this session. This field is set at session creation and cannot be changed with session.update.
    - type: :translation
      
      The session type. Always translation for Realtime translation sessions.
      - :translation
  - type: :"session.created"
    
    The event type, must be session.created.
    - :"session.created"
- class RealtimeTranslationSessionUpdatedEvent
  
  Returned when a translation session is updated with a session.update event, unless there is an error.
  - event_id: String
    
    The unique ID of the server event.
  - session: RealtimeTranslationSession
    
    The translation session configuration.
  - type: :"session.updated"
    
    The event type, must be session.updated.
    - :"session.updated"
- class RealtimeTranslationSessionClosedEvent
  
  Returned when a realtime translation session is closed.
  - event_id: String
    
    The unique ID of the server event.
  - type: :"session.closed"
    
    The event type, must be session.closed.
    - :"session.closed"
- class RealtimeTranslationInputTranscriptDeltaEvent
  
  Returned when optional source-language transcript text is available. This event is emitted only when audio.input.transcription is configured.
  
  Transcript deltas are append-only text fragments. Clients should not insert unconditional spaces between deltas.
  - delta: String
    
    Append-only source-language transcript text.
  - event_id: String
    
    The unique ID of the server event.
  - type: :"session.input_transcript.delta"
    
    The event type, must be session.input_transcript.delta.
    - :"session.input_transcript.delta"
  - elapsed_ms: Integer
    
    Timing metadata for stream alignment, derived from the translation frame when available. It advances in 200 ms increments, but multiple transcript deltas may share the same elapsed_ms. Treat it as alignment metadata, not a unique transcript-delta identifier.
- class RealtimeTranslationOutputTranscriptDeltaEvent
  
  Returned when translated transcript text is available.
  
  Transcript deltas are append-only text fragments. Clients should not insert unconditional spaces between deltas.
  - delta: String
    
    Append-only transcript text for the translated output audio.
  - event_id: String
    
    The unique ID of the server event.
  - type: :"session.output_transcript.delta"
    
    The event type, must be session.output_transcript.delta.
    - :"session.output_transcript.delta"
  - elapsed_ms: Integer
    
    Timing metadata for stream alignment, derived from the translation frame when available. It advances in 200 ms increments, but multiple transcript deltas may share the same elapsed_ms. Treat it as alignment metadata, not a unique transcript-delta identifier.
- class RealtimeTranslationOutputAudioDeltaEvent
  
  Returned when translated output audio is available. Output audio deltas are 200 ms frames of PCM16 audio.
  - delta: String
    
    Base64-encoded translated audio data.
  - event_id: String
    
    The unique ID of the server event.
  - type: :"session.output_audio.delta"
    
    The event type, must be session.output_audio.delta.
    - :"session.output_audio.delta"
  - channels: Integer
    
    Number of audio channels.
  - elapsed_ms: Integer
    
    Timing metadata for stream alignment, derived from the translation frame when available. Treat elapsed_ms as alignment metadata, not a unique event identifier.
  - format_: :pcm16
    
    Audio encoding for delta.
    - :pcm16
  - sample_rate: Integer
    
    Sample rate of the audio delta.

Realtime Translation Session

class RealtimeTranslationSession

A Realtime translation session. Translation sessions continuously translate input audio into the configured output language.
- id: String
  
  Unique identifier for the session that looks like sess_1234567890abcdef.
- audio: Audio{ input, output}
  
  Configuration for translation input and output audio.
  - input: Input{ noise_reduction, transcription}
    - noise_reduction: NoiseReduction{ type}
      
      Optional input noise reduction.
      - type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
    - transcription: Transcription{ model}
      
      Optional source-language transcription. When configured, the server emits session.input_transcript.delta events. Translation itself still runs from the input audio stream.
      - model: String
        
        The transcription model used for source transcript deltas.
  - output: Output{ language}
    - language: String
      
      Target language for translated output audio and transcript deltas.
- expires_at: Integer
  
  Expiration timestamp for the session, in seconds since epoch.
- model: String
  
  The Realtime translation model used for this session. This field is set at session creation and cannot be changed with session.update.
- type: :translation
  
  The session type. Always translation for Realtime translation sessions.
  - :translation

Realtime Translation Session Close Event

class RealtimeTranslationSessionCloseEvent

Gracefully close the realtime translation session. The server flushes pending input audio and emits any remaining translated output before closing the session.
- type: :"session.close"
  
  The event type, must be session.close.
  - :"session.close"
- event_id: String
  
  Optional client-generated ID used to identify this event.

Realtime Translation Session Closed Event

class RealtimeTranslationSessionClosedEvent

Returned when a realtime translation session is closed.
- event_id: String
  
  The unique ID of the server event.
- type: :"session.closed"
  
  The event type, must be session.closed.
  - :"session.closed"

Realtime Translation Session Create Request

class RealtimeTranslationSessionCreateRequest

Realtime translation session configuration. Translation sessions stream source audio in and translated audio plus transcript deltas out continuously.
- model: String
  
  The Realtime translation model used for this session.
- audio: Audio{ input, output}
  
  Configuration for translation input and output audio.
  - input: Input{ noise_reduction, transcription}
    - noise_reduction: NoiseReduction{ type}
      
      Optional input noise reduction. Set to null to disable it.
      - type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
    - transcription: Transcription{ model}
      
      Optional source-language transcription. When configured, the server emits session.input_transcript.delta events. Translation itself still runs from the input audio stream.
      - model: String
        
        The transcription model to use for source transcript deltas.
  - output: Output{ language}
    - language: String
      
      Target language for translated output audio and transcript deltas.

Realtime Translation Session Created Event

class RealtimeTranslationSessionCreatedEvent

Returned when a translation session is created. Emitted automatically when a new connection is established as the first server event. This event contains the default translation session configuration.
- event_id: String
  
  The unique ID of the server event.
- session: RealtimeTranslationSession
  
  The translation session configuration.
  - id: String
    
    Unique identifier for the session that looks like sess_1234567890abcdef.
  - audio: Audio{ input, output}
    
    Configuration for translation input and output audio.
    - input: Input{ noise_reduction, transcription}
      - noise_reduction: NoiseReduction{ type}
        
        Optional input noise reduction.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
      - transcription: Transcription{ model}
        
        Optional source-language transcription. When configured, the server emits session.input_transcript.delta events. Translation itself still runs from the input audio stream.
        
        model: String
        
        The transcription model used for source transcript deltas.
    - output: Output{ language}
      - language: String
        
        Target language for translated output audio and transcript deltas.
  - expires_at: Integer
    
    Expiration timestamp for the session, in seconds since epoch.
  - model: String
    
    The Realtime translation model used for this session. This field is set at session creation and cannot be changed with session.update.
  - type: :translation
    
    The session type. Always translation for Realtime translation sessions.
    - :translation
- type: :"session.created"
  
  The event type, must be session.created.
  - :"session.created"

Realtime Translation Session Update Event

class RealtimeTranslationSessionUpdateEvent

Send this event to update the translation session configuration. Translation sessions support updates to audio.output.language, audio.input.transcription, and audio.input.noise_reduction.
- session: RealtimeTranslationSessionUpdateRequest
  
  Translation session fields to update. The session type and model are set at creation and cannot be changed with session.update.
  - audio: Audio{ input, output}
    
    Configuration for translation input and output audio.
    - input: Input{ noise_reduction, transcription}
      - noise_reduction: NoiseReduction{ type}
        
        Optional input noise reduction. Set to null to disable it.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
      - transcription: Transcription{ model}
        
        Optional source-language transcription. When configured, the server emits session.input_transcript.delta events. Translation itself still runs from the input audio stream.
        
        model: String
        
        The transcription model to use for source transcript deltas.
    - output: Output{ language}
      - language: String
        
        Target language for translated output audio and transcript deltas.
- type: :"session.update"
  
  The event type, must be session.update.
  - :"session.update"
- event_id: String
  
  Optional client-generated ID used to identify this event.

Realtime Translation Session Update Request

class RealtimeTranslationSessionUpdateRequest

Realtime translation session fields that can be updated with session.update.
- audio: Audio{ input, output}
  
  Configuration for translation input and output audio.
  - input: Input{ noise_reduction, transcription}
    - noise_reduction: NoiseReduction{ type}
      
      Optional input noise reduction. Set to null to disable it.
      - type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
    - transcription: Transcription{ model}
      
      Optional source-language transcription. When configured, the server emits session.input_transcript.delta events. Translation itself still runs from the input audio stream.
      - model: String
        
        The transcription model to use for source transcript deltas.
  - output: Output{ language}
    - language: String
      
      Target language for translated output audio and transcript deltas.

Realtime Translation Session Updated Event

class RealtimeTranslationSessionUpdatedEvent

Returned when a translation session is updated with a session.update event, unless there is an error.
- event_id: String
  
  The unique ID of the server event.
- session: RealtimeTranslationSession
  
  The translation session configuration.
  - id: String
    
    Unique identifier for the session that looks like sess_1234567890abcdef.
  - audio: Audio{ input, output}
    
    Configuration for translation input and output audio.
    - input: Input{ noise_reduction, transcription}
      - noise_reduction: NoiseReduction{ type}
        
        Optional input noise reduction.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
      - transcription: Transcription{ model}
        
        Optional source-language transcription. When configured, the server emits session.input_transcript.delta events. Translation itself still runs from the input audio stream.
        
        model: String
        
        The transcription model used for source transcript deltas.
    - output: Output{ language}
      - language: String
        
        Target language for translated output audio and transcript deltas.
  - expires_at: Integer
    
    Expiration timestamp for the session, in seconds since epoch.
  - model: String
    
    The Realtime translation model used for this session. This field is set at session creation and cannot be changed with session.update.
  - type: :translation
    
    The session type. Always translation for Realtime translation sessions.
    - :translation
- type: :"session.updated"
  
  The event type, must be session.updated.
  - :"session.updated"

Realtime Truncation

RealtimeTruncation = :auto | :disabled | RealtimeTruncationRetentionRatio

When the number of tokens in a conversation exceeds the model's input token limit, the conversation be truncated, meaning messages (starting from the oldest) will not be included in the model's context. A 32k context model with 4,096 max output tokens can only include 28,224 tokens in the context before truncation occurs.

Clients can configure truncation behavior to truncate with a lower max token limit, which is an effective way to control token usage and cost.

Truncation will reduce the number of cached tokens on the next turn (busting the cache), since messages are dropped from the beginning of the context. However, clients can also configure truncation to retain messages up to a fraction of the maximum context size, which will reduce the need for future truncations and thus improve the cache rate.

Truncation can be disabled entirely, which means the server will never truncate but would instead return an error if the conversation exceeds the model's input token limit.
- RealtimeTruncationStrategy = :auto | :disabled
  
  The truncation strategy to use for the session. auto is the default truncation strategy. disabled will disable truncation and emit errors when the conversation exceeds the input token limit.
  - :auto
  - :disabled
- class RealtimeTruncationRetentionRatio
  
  Retain a fraction of the conversation tokens when the conversation exceeds the input token limit. This allows you to amortize truncations across multiple turns, which can help improve cached token usage.
  - retention_ratio: Float
    
    Fraction of post-instruction conversation tokens to retain (0.0 - 1.0) when the conversation exceeds the input token limit. Setting this to 0.8 means that messages will be dropped until 80% of the maximum allowed tokens are used. This helps reduce the frequency of truncations and improve cache rates.
  - type: :retention_ratio
    
    Use retention ratio truncation.
    - :retention_ratio
  - token_limits: TokenLimits{ post_instructions}
    
    Optional custom token limits for this truncation strategy. If not provided, the model's default token limits will be used.
    - post_instructions: Integer
      
      Maximum tokens allowed in the conversation after instructions (which including tool definitions). For example, setting this to 5,000 would mean that truncation would occur when the conversation exceeds 5,000 tokens after instructions. This cannot be higher than the model's context window size minus the maximum output tokens.

Realtime Truncation Retention Ratio

class RealtimeTruncationRetentionRatio

Retain a fraction of the conversation tokens when the conversation exceeds the input token limit. This allows you to amortize truncations across multiple turns, which can help improve cached token usage.
- retention_ratio: Float
  
  Fraction of post-instruction conversation tokens to retain (0.0 - 1.0) when the conversation exceeds the input token limit. Setting this to 0.8 means that messages will be dropped until 80% of the maximum allowed tokens are used. This helps reduce the frequency of truncations and improve cache rates.
- type: :retention_ratio
  
  Use retention ratio truncation.
  - :retention_ratio
- token_limits: TokenLimits{ post_instructions}
  
  Optional custom token limits for this truncation strategy. If not provided, the model's default token limits will be used.
  - post_instructions: Integer
    
    Maximum tokens allowed in the conversation after instructions (which including tool definitions). For example, setting this to 5,000 would mean that truncation would occur when the conversation exceeds 5,000 tokens after instructions. This cannot be higher than the model's context window size minus the maximum output tokens.

Response Audio Delta Event

class ResponseAudioDeltaEvent

Returned when the model-generated audio is updated.
- content_index: Integer
  
  The index of the content part in the item's content array.
- delta: String
  
  Base64-encoded audio data delta.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the item.
- output_index: Integer
  
  The index of the output item in the response.
- response_id: String
  
  The ID of the response.
- type: :"response.output_audio.delta"
  
  The event type, must be response.output_audio.delta.
  - :"response.output_audio.delta"

Response Audio Done Event

class ResponseAudioDoneEvent

Returned when the model-generated audio is done. Also emitted when a Response is interrupted, incomplete, or cancelled.
- content_index: Integer
  
  The index of the content part in the item's content array.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the item.
- output_index: Integer
  
  The index of the output item in the response.
- response_id: String
  
  The ID of the response.
- type: :"response.output_audio.done"
  
  The event type, must be response.output_audio.done.
  - :"response.output_audio.done"

Response Audio Transcript Delta Event

class ResponseAudioTranscriptDeltaEvent

Returned when the model-generated transcription of audio output is updated.
- content_index: Integer
  
  The index of the content part in the item's content array.
- delta: String
  
  The transcript delta.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the item.
- output_index: Integer
  
  The index of the output item in the response.
- response_id: String
  
  The ID of the response.
- type: :"response.output_audio_transcript.delta"
  
  The event type, must be response.output_audio_transcript.delta.
  - :"response.output_audio_transcript.delta"

Response Audio Transcript Done Event

class ResponseAudioTranscriptDoneEvent

Returned when the model-generated transcription of audio output is done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.
- content_index: Integer
  
  The index of the content part in the item's content array.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the item.
- output_index: Integer
  
  The index of the output item in the response.
- response_id: String
  
  The ID of the response.
- transcript: String
  
  The final transcript of the audio.
- type: :"response.output_audio_transcript.done"
  
  The event type, must be response.output_audio_transcript.done.
  - :"response.output_audio_transcript.done"

Response Cancel Event

class ResponseCancelEvent

Send this event to cancel an in-progress response. The server will respond with a response.done event with a status of response.status=cancelled. If there is no response to cancel, the server will respond with an error. It's safe to call response.cancel even if no response is in progress, an error will be returned the session will remain unaffected.
- type: :"response.cancel"
  
  The event type, must be response.cancel.
  - :"response.cancel"
- event_id: String
  
  Optional client-generated ID used to identify this event.
- response_id: String
  
  A specific response ID to cancel - if not provided, will cancel an in-progress response in the default conversation.

Response Content Part Added Event

class ResponseContentPartAddedEvent

Returned when a new content part is added to an assistant message item during response generation.
- content_index: Integer
  
  The index of the content part in the item's content array.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the item to which the content part was added.
- output_index: Integer
  
  The index of the output item in the response.
- part: Part{ audio, text, transcript, type}
  
  The content part that was added.
  - audio: String
    
    Base64-encoded audio data (if type is "audio").
  - text: String
    
    The text content (if type is "text").
  - transcript: String
    
    The transcript of the audio (if type is "audio").
  - type: :text | :audio
    
    The content type ("text", "audio").
    - :text
    - :audio
- response_id: String
  
  The ID of the response.
- type: :"response.content_part.added"
  
  The event type, must be response.content_part.added.
  - :"response.content_part.added"

Response Content Part Done Event

class ResponseContentPartDoneEvent

Returned when a content part is done streaming in an assistant message item. Also emitted when a Response is interrupted, incomplete, or cancelled.
- content_index: Integer
  
  The index of the content part in the item's content array.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the item.
- output_index: Integer
  
  The index of the output item in the response.
- part: Part{ audio, text, transcript, type}
  
  The content part that is done.
  - audio: String
    
    Base64-encoded audio data (if type is "audio").
  - text: String
    
    The text content (if type is "text").
  - transcript: String
    
    The transcript of the audio (if type is "audio").
  - type: :text | :audio
    
    The content type ("text", "audio").
    - :text
    - :audio
- response_id: String
  
  The ID of the response.
- type: :"response.content_part.done"
  
  The event type, must be response.content_part.done.
  - :"response.content_part.done"

Response Create Event

class ResponseCreateEvent

This event instructs the server to create a Response, which means triggering model inference. When in Server VAD mode, the server will create Responses automatically.

A Response will include at least one Item, and may have two, in which case the second will be a function call. These Items will be appended to the conversation history by default.

The server will respond with a response.created event, events for Items and content created, and finally a response.done event to indicate the Response is complete.

The response.create event includes inference configuration like instructions and tools. If these are set, they will override the Session's configuration for this Response only.

Responses can be created out-of-band of the default Conversation, meaning that they can have arbitrary input, and it's possible to disable writing the output to the Conversation. Only one Response can write to the default Conversation at a time, but otherwise multiple Responses can be created in parallel. The metadata field is a good way to disambiguate multiple simultaneous Responses.

Clients can set conversation to none to create a Response that does not write to the default Conversation. Arbitrary input can be provided with the input field, which is an array accepting raw Items and references to existing Items.
- type: :"response.create"
  
  The event type, must be response.create.
  - :"response.create"
- event_id: String
  
  Optional client-generated ID used to identify this event.
- response: RealtimeResponseCreateParams
  
  Create a new Realtime response with these parameters
  - audio: RealtimeResponseCreateAudioOutput
    
    Configuration for audio input and output.
    - output: Output{ format_, voice}
      - format_: RealtimeAudioFormats
        
        The format of the output audio.
        
        class AudioPCM
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
        
        type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
        
        class AudioPCMU
        
        The G.711 μ-law format.
        
        type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
        
        class AudioPCMA
        
        The G.711 A-law format.
        
        type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
      - voice: String | :alloy | :ash | :ballad | 7 more | ID{ id}
        
        The voice the model uses to respond. Supported built-in voices are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. You may also provide a custom voice object with an id, for example { "id": "voice_1234" }. Voice cannot be changed during the session once the model has responded with audio at least once. We recommend marin and cedar for best quality.
        
        String = String
        
        Voice = :alloy | :ash | :ballad | 7 more
        
        :alloy
        
        :ash
        
        :ballad
        
        :coral
        
        :echo
        
        :sage
        
        :shimmer
        
        :verse
        
        :marin
        
        :cedar
        
        class ID
        
        Custom voice reference.
        
        id: String
        
        The custom voice ID, e.g. voice_1234.
  - conversation: String | :auto | :none
    
    Controls which conversation the response is added to. Currently supports auto and none, with auto as the default value. The auto value means that the contents of the response will be added to the default conversation. Set this to none to create an out-of-band response which will not add items to default conversation.
    - String = String
    - Conversation = :auto | :none
      
      Controls which conversation the response is added to. Currently supports auto and none, with auto as the default value. The auto value means that the contents of the response will be added to the default conversation. Set this to none to create an out-of-band response which will not add items to default conversation.
      - :auto
      - :none
  - input: Array[ConversationItem]
    
    Input items to include in the prompt for the model. Using this field creates a new context for this Response instead of using the default conversation. An empty array [] will clear the context for this Response. Note that this can include references to items that previously appeared in the session using their id.
    - class RealtimeConversationItemSystemMessage
      
      A system message in a Realtime conversation can be used to provide additional context or instructions to the model. This is similar but distinct from the instruction prompt provided at the start of a conversation, as system messages can be added at any point in the conversation. For major changes to the conversation's behavior, use instructions, but for smaller updates (e.g. "the user is now asking about a different topic"), use system messages.
      - content: Array[Content{ text, type}]
        
        The content of the message.
        
        text: String
        
        The text content.
        
        type: :input_text
        
        The content type. Always input_text for system messages.
        
        :input_text
      - role: :system
        
        The role of the message sender. Always system.
        
        :system
      - type: :message
        
        The type of the item. Always message.
        
        :message
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeConversationItemUserMessage
      
      A user message item in a Realtime conversation.
      - content: Array[Content{ audio, detail, image_url, 3 more}]
        
        The content of the message.
        
        audio: String
        
        Base64-encoded audio bytes (for input_audio), these will be parsed as the format specified in the session input audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
        
        detail: :auto | :low | :high
        
        The detail level of the image (for input_image). auto will default to high.
        
        :auto
        
        :low
        
        :high
        
        image_url: String
        
        Base64-encoded image bytes (for input_image) as a data URI. For example data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA.... Supported formats are PNG and JPEG.
        
        text: String
        
        The text content (for input_text).
        
        transcript: String
        
        Transcript of the audio (for input_audio). This is not sent to the model, but will be attached to the message item for reference.
        
        type: :input_text | :input_audio | :input_image
        
        The content type (input_text, input_audio, or input_image).
        
        :input_text
        
        :input_audio
        
        :input_image
      - role: :user
        
        The role of the message sender. Always user.
        
        :user
      - type: :message
        
        The type of the item. Always message.
        
        :message
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeConversationItemAssistantMessage
      
      An assistant message item in a Realtime conversation.
      - content: Array[Content{ audio, text, transcript, type}]
        
        The content of the message.
        
        audio: String
        
        Base64-encoded audio bytes, these will be parsed as the format specified in the session output audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
        
        text: String
        
        The text content.
        
        transcript: String
        
        The transcript of the audio content, this will always be present if the output type is audio.
        
        type: :output_text | :output_audio
        
        The content type, output_text or output_audio depending on the session output_modalities configuration.
        
        :output_text
        
        :output_audio
      - role: :assistant
        
        The role of the message sender. Always assistant.
        
        :assistant
      - type: :message
        
        The type of the item. Always message.
        
        :message
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeConversationItemFunctionCall
      
      A function call item in a Realtime conversation.
      - arguments: String
        
        The arguments of the function call. This is a JSON-encoded string representing the arguments passed to the function, for example {"arg1": "value1", "arg2": 42}.
      - name: String
        
        The name of the function being called.
      - type: :function_call
        
        The type of the item. Always function_call.
        
        :function_call
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - call_id: String
        
        The ID of the function call.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeConversationItemFunctionCallOutput
      
      A function call output item in a Realtime conversation.
      - call_id: String
        
        The ID of the function call this output is for.
      - output: String
        
        The output of the function call, this is free text and can contain any information or simply be empty.
      - type: :function_call_output
        
        The type of the item. Always function_call_output.
        
        :function_call_output
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeMcpApprovalResponse
      
      A Realtime item responding to an MCP approval request.
      - id: String
        
        The unique ID of the approval response.
      - approval_request_id: String
        
        The ID of the approval request being answered.
      - approve: bool
        
        Whether the request was approved.
      - type: :mcp_approval_response
        
        The type of the item. Always mcp_approval_response.
        
        :mcp_approval_response
      - reason: String
        
        Optional reason for the decision.
    - class RealtimeMcpListTools
      
      A Realtime item listing tools available on an MCP server.
      - server_label: String
        
        The label of the MCP server.
      - tools: Array[Tool{ input_schema, name, annotations, description}]
        
        The tools available on the server.
        
        input_schema: untyped
        
        The JSON schema describing the tool's input.
        
        name: String
        
        The name of the tool.
        
        annotations: untyped
        
        Additional annotations about the tool.
        
        description: String
        
        The description of the tool.
      - type: :mcp_list_tools
        
        The type of the item. Always mcp_list_tools.
        
        :mcp_list_tools
      - id: String
        
        The unique ID of the list.
    - class RealtimeMcpToolCall
      
      A Realtime item representing an invocation of a tool on an MCP server.
      - id: String
        
        The unique ID of the tool call.
      - arguments: String
        
        A JSON string of the arguments passed to the tool.
      - name: String
        
        The name of the tool that was run.
      - server_label: String
        
        The label of the MCP server running the tool.
      - type: :mcp_call
        
        The type of the item. Always mcp_call.
        
        :mcp_call
      - approval_request_id: String
        
        The ID of an associated approval request, if any.
      - error: RealtimeMcpProtocolError | RealtimeMcpToolExecutionError | RealtimeMcphttpError
        
        The error from the tool call, if any.
        
        class RealtimeMcpProtocolError
        
        code: Integer
        
        message: String
        
        type: :protocol_error
        
        :protocol_error
        
        class RealtimeMcpToolExecutionError
        
        message: String
        
        type: :tool_execution_error
        
        :tool_execution_error
        
        class RealtimeMcphttpError
        
        code: Integer
        
        message: String
        
        type: :http_error
        
        :http_error
      - output: String
        
        The output from the tool call.
    - class RealtimeMcpApprovalRequest
      
      A Realtime item requesting human approval of a tool invocation.
      - id: String
        
        The unique ID of the approval request.
      - arguments: String
        
        A JSON string of arguments for the tool.
      - name: String
        
        The name of the tool to run.
      - server_label: String
        
        The label of the MCP server making the request.
      - type: :mcp_approval_request
        
        The type of the item. Always mcp_approval_request.
        
        :mcp_approval_request
  - instructions: String
    
    The default system instructions (i.e. system message) prepended to model calls. This field allows the client to guide the model on desired responses. The model can be instructed on response content and format, (e.g. "be extremely succinct", "act friendly", "here are examples of good responses") and on audio behavior (e.g. "talk quickly", "inject emotion into your voice", "laugh frequently"). The instructions are not guaranteed to be followed by the model, but they provide guidance to the model on the desired behavior. Note that the server sets default instructions which will be used if this field is not set and are visible in the session.created event at the start of the session.
  - max_output_tokens: Integer | :inf
    
    Maximum number of output tokens for a single assistant response, inclusive of tool calls. Provide an integer between 1 and 4096 to limit output tokens, or inf for the maximum available tokens for a given model. Defaults to inf.
    - Integer = Integer
    - MaxOutputTokens = :inf
      - :inf
  - metadata: Metadata
    
    Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard.
    
    Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.
  - output_modalities: Array[:text | :audio]
    
    The set of modalities the model used to respond, currently the only possible values are [\"audio\"], [\"text\"]. Audio output always include a text transcript. Setting the output to mode text will disable audio output from the model.
    - :text
    - :audio
  - parallel_tool_calls: bool
    
    Whether the model may call multiple tools in parallel. Only supported by reasoning Realtime models such as gpt-realtime-2.
  - prompt: ResponsePrompt
    
    Reference to a prompt template and its variables. Learn more.
    - id: String
      
      The unique identifier of the prompt template to use.
    - variables: Hash[Symbol, String | ResponseInputText | ResponseInputImage | ResponseInputFile]
      
      Optional map of values to substitute in for variables in your prompt. The substitution values can either be strings, or other Response input types like images or files.
      - String = String
      - class ResponseInputText
        
        A text input to the model.
        
        text: String
        
        The text input to the model.
        
        type: :input_text
        
        The type of the input item. Always input_text.
        
        :input_text
      - class ResponseInputImage
        
        An image input to the model. Learn about image inputs.
        
        detail: :low | :high | :auto | :original
        
        The detail level of the image to be sent to the model. One of high, low, auto, or original. Defaults to auto.
        
        :low
        
        :high
        
        :auto
        
        :original
        
        type: :input_image
        
        The type of the input item. Always input_image.
        
        :input_image
        
        file_id: String
        
        The ID of the file to be sent to the model.
        
        image_url: String
        
        The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.
      - class ResponseInputFile
        
        A file input to the model.
        
        type: :input_file
        
        The type of the input item. Always input_file.
        
        :input_file
        
        detail: :low | :high
        
        The detail level of the file to be sent to the model. Use low for the default rendering behavior, or high to render the file at higher quality. Defaults to low.
        
        :low
        
        :high
        
        file_data: String
        
        The content of the file to be sent to the model.
        
        file_id: String
        
        The ID of the file to be sent to the model.
        
        file_url: String
        
        The URL of the file to be sent to the model.
        
        filename: String
        
        The name of the file to be sent to the model.
    - version: String
      
      Optional version of the prompt template.
  - reasoning: RealtimeReasoning
    
    Configuration for reasoning-capable Realtime models such as gpt-realtime-2.
    - effort: RealtimeReasoningEffort
      
      Constrains effort on reasoning for reasoning-capable Realtime models such as gpt-realtime-2.
      - :minimal
      - :low
      - :medium
      - :high
      - :xhigh
  - tool_choice: ToolChoiceOptions | ToolChoiceFunction | ToolChoiceMcp
    
    How the model chooses tools. Provide one of the string modes or force a specific function/MCP tool.
    - ToolChoiceOptions = :none | :auto | :required
      
      Controls which (if any) tool is called by the model.
      
      none means the model will not call any tool and instead generates a message.
      
      auto means the model can pick between generating a message or calling one or more tools.
      
      required means the model must call one or more tools.
      - :none
      - :auto
      - :required
    - class ToolChoiceFunction
      
      Use this option to force the model to call a specific function.
      - name: String
        
        The name of the function to call.
      - type: :function
        
        For function calling, the type is always function.
        
        :function
    - class ToolChoiceMcp
      
      Use this option to force the model to call a specific tool on a remote MCP server.
      - server_label: String
        
        The label of the MCP server to use.
      - type: :mcp
        
        For MCP tools, the type is always mcp.
        
        :mcp
      - name: String
        
        The name of the tool to call on the server.
  - tools: Array[RealtimeFunctionTool | RealtimeResponseCreateMcpTool]
    
    Tools available to the model.
    - class RealtimeFunctionTool
      - description: String
        
        The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything).
      - name: String
        
        The name of the function.
      - parameters: untyped
        
        Parameters of the function in JSON Schema.
      - type: :function
        
        The type of the tool, i.e. function.
        
        :function
    - class RealtimeResponseCreateMcpTool
      
      Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.
      - server_label: String
        
        A label for this MCP server, used to identify it in tool calls.
      - type: :mcp
        
        The type of the MCP tool. Always mcp.
        
        :mcp
      - allowed_tools: Array[String] | McpToolFilter{ read_only, tool_names}
        
        List of allowed tool names or a filter object.
        
        McpAllowedTools = Array[String]
        
        A string array of allowed tool names
        
        class McpToolFilter
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
      - authorization: String
        
        An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.
      - connector_id: :connector_dropbox | :connector_gmail | :connector_googlecalendar | 5 more
        
        Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here.
        
        Currently supported connector_id values are:
        
        Dropbox: connector_dropbox
        
        Gmail: connector_gmail
        
        Google Calendar: connector_googlecalendar
        
        Google Drive: connector_googledrive
        
        Microsoft Teams: connector_microsoftteams
        
        Outlook Calendar: connector_outlookcalendar
        
        Outlook Email: connector_outlookemail
        
        SharePoint: connector_sharepoint
        
        :connector_dropbox
        
        :connector_gmail
        
        :connector_googlecalendar
        
        :connector_googledrive
        
        :connector_microsoftteams
        
        :connector_outlookcalendar
        
        :connector_outlookemail
        
        :connector_sharepoint
      - defer_loading: bool
        
        Whether this MCP tool is deferred and discovered via tool search.
      - headers: Hash[Symbol, String]
        
        Optional HTTP headers to send to the MCP server. Use for authentication or other purposes.
      - require_approval: McpToolApprovalFilter{ always, never} | :always | :never
        
        Specify which of the MCP server's tools require approval.
        
        class McpToolApprovalFilter
        
        Specify which of the MCP server's tools require approval. Can be always, never, or a filter object associated with tools that require approval.
        
        always: Always{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        never: Never{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        McpToolApprovalSetting = :always | :never
        
        Specify a single approval policy for all tools. One of always or never. When set to always, all tools will require approval. When set to never, all tools will not require approval.
        
        :always
        
        :never
      - server_description: String
        
        Optional description of the MCP server, used to provide more context.
      - server_url: String
        
        The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided.
      - tunnel_id: String
        
        The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided.

Response Created Event

class ResponseCreatedEvent

Returned when a new Response is created. The first event of response creation, where the response is in an initial state of in_progress.
- event_id: String
  
  The unique ID of the server event.
- response: RealtimeResponse
  
  The response resource.
  - id: String
    
    The unique ID of the response, will look like resp_1234.
  - audio: Audio{ output}
    
    Configuration for audio output.
    - output: Output{ format_, voice}
      - format_: RealtimeAudioFormats
        
        The format of the output audio.
        
        class AudioPCM
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
        
        type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
        
        class AudioPCMU
        
        The G.711 μ-law format.
        
        type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
        
        class AudioPCMA
        
        The G.711 A-law format.
        
        type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
      - voice: String | :alloy | :ash | :ballad | 7 more
        
        The voice the model uses to respond. Voice cannot be changed during the session once the model has responded with audio at least once. Current voice options are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. We recommend marin and cedar for best quality.
        
        String = String
        
        Voice = :alloy | :ash | :ballad | 7 more
        
        The voice the model uses to respond. Voice cannot be changed during the session once the model has responded with audio at least once. Current voice options are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. We recommend marin and cedar for best quality.
        
        :alloy
        
        :ash
        
        :ballad
        
        :coral
        
        :echo
        
        :sage
        
        :shimmer
        
        :verse
        
        :marin
        
        :cedar
  - conversation_id: String
    
    Which conversation the response is added to, determined by the conversation field in the response.create event. If auto, the response will be added to the default conversation and the value of conversation_id will be an id like conv_1234. If none, the response will not be added to any conversation and the value of conversation_id will be null. If responses are being triggered automatically by VAD the response will be added to the default conversation
  - max_output_tokens: Integer | :inf
    
    Maximum number of output tokens for a single assistant response, inclusive of tool calls, that was used in this response.
    - Integer = Integer
    - MaxOutputTokens = :inf
      - :inf
  - metadata: Metadata
    
    Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard.
    
    Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.
  - object: :"realtime.response"
    
    The object type, must be realtime.response.
    - :"realtime.response"
  - output: Array[ConversationItem]
    
    The list of output items generated by the response.
    - class RealtimeConversationItemSystemMessage
      
      A system message in a Realtime conversation can be used to provide additional context or instructions to the model. This is similar but distinct from the instruction prompt provided at the start of a conversation, as system messages can be added at any point in the conversation. For major changes to the conversation's behavior, use instructions, but for smaller updates (e.g. "the user is now asking about a different topic"), use system messages.
      - content: Array[Content{ text, type}]
        
        The content of the message.
        
        text: String
        
        The text content.
        
        type: :input_text
        
        The content type. Always input_text for system messages.
        
        :input_text
      - role: :system
        
        The role of the message sender. Always system.
        
        :system
      - type: :message
        
        The type of the item. Always message.
        
        :message
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeConversationItemUserMessage
      
      A user message item in a Realtime conversation.
      - content: Array[Content{ audio, detail, image_url, 3 more}]
        
        The content of the message.
        
        audio: String
        
        Base64-encoded audio bytes (for input_audio), these will be parsed as the format specified in the session input audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
        
        detail: :auto | :low | :high
        
        The detail level of the image (for input_image). auto will default to high.
        
        :auto
        
        :low
        
        :high
        
        image_url: String
        
        Base64-encoded image bytes (for input_image) as a data URI. For example data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA.... Supported formats are PNG and JPEG.
        
        text: String
        
        The text content (for input_text).
        
        transcript: String
        
        Transcript of the audio (for input_audio). This is not sent to the model, but will be attached to the message item for reference.
        
        type: :input_text | :input_audio | :input_image
        
        The content type (input_text, input_audio, or input_image).
        
        :input_text
        
        :input_audio
        
        :input_image
      - role: :user
        
        The role of the message sender. Always user.
        
        :user
      - type: :message
        
        The type of the item. Always message.
        
        :message
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeConversationItemAssistantMessage
      
      An assistant message item in a Realtime conversation.
      - content: Array[Content{ audio, text, transcript, type}]
        
        The content of the message.
        
        audio: String
        
        Base64-encoded audio bytes, these will be parsed as the format specified in the session output audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
        
        text: String
        
        The text content.
        
        transcript: String
        
        The transcript of the audio content, this will always be present if the output type is audio.
        
        type: :output_text | :output_audio
        
        The content type, output_text or output_audio depending on the session output_modalities configuration.
        
        :output_text
        
        :output_audio
      - role: :assistant
        
        The role of the message sender. Always assistant.
        
        :assistant
      - type: :message
        
        The type of the item. Always message.
        
        :message
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeConversationItemFunctionCall
      
      A function call item in a Realtime conversation.
      - arguments: String
        
        The arguments of the function call. This is a JSON-encoded string representing the arguments passed to the function, for example {"arg1": "value1", "arg2": 42}.
      - name: String
        
        The name of the function being called.
      - type: :function_call
        
        The type of the item. Always function_call.
        
        :function_call
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - call_id: String
        
        The ID of the function call.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeConversationItemFunctionCallOutput
      
      A function call output item in a Realtime conversation.
      - call_id: String
        
        The ID of the function call this output is for.
      - output: String
        
        The output of the function call, this is free text and can contain any information or simply be empty.
      - type: :function_call_output
        
        The type of the item. Always function_call_output.
        
        :function_call_output
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeMcpApprovalResponse
      
      A Realtime item responding to an MCP approval request.
      - id: String
        
        The unique ID of the approval response.
      - approval_request_id: String
        
        The ID of the approval request being answered.
      - approve: bool
        
        Whether the request was approved.
      - type: :mcp_approval_response
        
        The type of the item. Always mcp_approval_response.
        
        :mcp_approval_response
      - reason: String
        
        Optional reason for the decision.
    - class RealtimeMcpListTools
      
      A Realtime item listing tools available on an MCP server.
      - server_label: String
        
        The label of the MCP server.
      - tools: Array[Tool{ input_schema, name, annotations, description}]
        
        The tools available on the server.
        
        input_schema: untyped
        
        The JSON schema describing the tool's input.
        
        name: String
        
        The name of the tool.
        
        annotations: untyped
        
        Additional annotations about the tool.
        
        description: String
        
        The description of the tool.
      - type: :mcp_list_tools
        
        The type of the item. Always mcp_list_tools.
        
        :mcp_list_tools
      - id: String
        
        The unique ID of the list.
    - class RealtimeMcpToolCall
      
      A Realtime item representing an invocation of a tool on an MCP server.
      - id: String
        
        The unique ID of the tool call.
      - arguments: String
        
        A JSON string of the arguments passed to the tool.
      - name: String
        
        The name of the tool that was run.
      - server_label: String
        
        The label of the MCP server running the tool.
      - type: :mcp_call
        
        The type of the item. Always mcp_call.
        
        :mcp_call
      - approval_request_id: String
        
        The ID of an associated approval request, if any.
      - error: RealtimeMcpProtocolError | RealtimeMcpToolExecutionError | RealtimeMcphttpError
        
        The error from the tool call, if any.
        
        class RealtimeMcpProtocolError
        
        code: Integer
        
        message: String
        
        type: :protocol_error
        
        :protocol_error
        
        class RealtimeMcpToolExecutionError
        
        message: String
        
        type: :tool_execution_error
        
        :tool_execution_error
        
        class RealtimeMcphttpError
        
        code: Integer
        
        message: String
        
        type: :http_error
        
        :http_error
      - output: String
        
        The output from the tool call.
    - class RealtimeMcpApprovalRequest
      
      A Realtime item requesting human approval of a tool invocation.
      - id: String
        
        The unique ID of the approval request.
      - arguments: String
        
        A JSON string of arguments for the tool.
      - name: String
        
        The name of the tool to run.
      - server_label: String
        
        The label of the MCP server making the request.
      - type: :mcp_approval_request
        
        The type of the item. Always mcp_approval_request.
        
        :mcp_approval_request
  - output_modalities: Array[:text | :audio]
    
    The set of modalities the model used to respond, currently the only possible values are [\"audio\"], [\"text\"]. Audio output always include a text transcript. Setting the output to mode text will disable audio output from the model.
    - :text
    - :audio
  - status: :completed | :cancelled | :failed | 2 more
    
    The final status of the response (completed, cancelled, failed, or incomplete, in_progress).
    - :completed
    - :cancelled
    - :failed
    - :incomplete
    - :in_progress
  - status_details: RealtimeResponseStatus
    
    Additional details about the status.
    - error: Error{ code, type}
      
      A description of the error that caused the response to fail, populated when the status is failed.
      - code: String
        
        Error code, if any.
      - type: String
        
        The type of error.
    - reason: :turn_detected | :client_cancelled | :max_output_tokens | :content_filter
      
      The reason the Response did not complete. For a cancelled Response, one of turn_detected (the server VAD detected a new start of speech) or client_cancelled (the client sent a cancel event). For an incomplete Response, one of max_output_tokens or content_filter (the server-side safety filter activated and cut off the response).
      - :turn_detected
      - :client_cancelled
      - :max_output_tokens
      - :content_filter
    - type: :completed | :cancelled | :incomplete | :failed
      
      The type of error that caused the response to fail, corresponding with the status field (completed, cancelled, incomplete, failed).
      - :completed
      - :cancelled
      - :incomplete
      - :failed
  - usage: RealtimeResponseUsage
    
    Usage statistics for the Response, this will correspond to billing. A Realtime API session will maintain a conversation context and append new Items to the Conversation, thus output from previous turns (text and audio tokens) will become the input for later turns.
    - input_token_details: RealtimeResponseUsageInputTokenDetails
      
      Details about the input tokens used in the Response. Cached tokens are tokens from previous turns in the conversation that are included as context for the current response. Cached tokens here are counted as a subset of input tokens, meaning input tokens will include cached and uncached tokens.
      - audio_tokens: Integer
        
        The number of audio tokens used as input for the Response.
      - cached_tokens: Integer
        
        The number of cached tokens used as input for the Response.
      - cached_tokens_details: CachedTokensDetails{ audio_tokens, image_tokens, text_tokens}
        
        Details about the cached tokens used as input for the Response.
        
        audio_tokens: Integer
        
        The number of cached audio tokens used as input for the Response.
        
        image_tokens: Integer
        
        The number of cached image tokens used as input for the Response.
        
        text_tokens: Integer
        
        The number of cached text tokens used as input for the Response.
      - image_tokens: Integer
        
        The number of image tokens used as input for the Response.
      - text_tokens: Integer
        
        The number of text tokens used as input for the Response.
    - input_tokens: Integer
      
      The number of input tokens used in the Response, including text and audio tokens.
    - output_token_details: RealtimeResponseUsageOutputTokenDetails
      
      Details about the output tokens used in the Response.
      - audio_tokens: Integer
        
        The number of audio tokens used in the Response.
      - text_tokens: Integer
        
        The number of text tokens used in the Response.
    - output_tokens: Integer
      
      The number of output tokens sent in the Response, including text and audio tokens.
    - total_tokens: Integer
      
      The total number of tokens in the Response including input and output text and audio tokens.
- type: :"response.created"
  
  The event type, must be response.created.
  - :"response.created"

Response Done Event

class ResponseDoneEvent

Returned when a Response is done streaming. Always emitted, no matter the final state. The Response object included in the response.done event will include all output Items in the Response but will omit the raw audio data.

Clients should check the status field of the Response to determine if it was successful (completed) or if there was another outcome: cancelled, failed, or incomplete.

A response will contain all output items that were generated during the response, excluding any audio content.
- event_id: String
  
  The unique ID of the server event.
- response: RealtimeResponse
  
  The response resource.
  - id: String
    
    The unique ID of the response, will look like resp_1234.
  - audio: Audio{ output}
    
    Configuration for audio output.
    - output: Output{ format_, voice}
      - format_: RealtimeAudioFormats
        
        The format of the output audio.
        
        class AudioPCM
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
        
        type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
        
        class AudioPCMU
        
        The G.711 μ-law format.
        
        type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
        
        class AudioPCMA
        
        The G.711 A-law format.
        
        type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
      - voice: String | :alloy | :ash | :ballad | 7 more
        
        The voice the model uses to respond. Voice cannot be changed during the session once the model has responded with audio at least once. Current voice options are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. We recommend marin and cedar for best quality.
        
        String = String
        
        Voice = :alloy | :ash | :ballad | 7 more
        
        The voice the model uses to respond. Voice cannot be changed during the session once the model has responded with audio at least once. Current voice options are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. We recommend marin and cedar for best quality.
        
        :alloy
        
        :ash
        
        :ballad
        
        :coral
        
        :echo
        
        :sage
        
        :shimmer
        
        :verse
        
        :marin
        
        :cedar
  - conversation_id: String
    
    Which conversation the response is added to, determined by the conversation field in the response.create event. If auto, the response will be added to the default conversation and the value of conversation_id will be an id like conv_1234. If none, the response will not be added to any conversation and the value of conversation_id will be null. If responses are being triggered automatically by VAD the response will be added to the default conversation
  - max_output_tokens: Integer | :inf
    
    Maximum number of output tokens for a single assistant response, inclusive of tool calls, that was used in this response.
    - Integer = Integer
    - MaxOutputTokens = :inf
      - :inf
  - metadata: Metadata
    
    Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard.
    
    Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.
  - object: :"realtime.response"
    
    The object type, must be realtime.response.
    - :"realtime.response"
  - output: Array[ConversationItem]
    
    The list of output items generated by the response.
    - class RealtimeConversationItemSystemMessage
      
      A system message in a Realtime conversation can be used to provide additional context or instructions to the model. This is similar but distinct from the instruction prompt provided at the start of a conversation, as system messages can be added at any point in the conversation. For major changes to the conversation's behavior, use instructions, but for smaller updates (e.g. "the user is now asking about a different topic"), use system messages.
      - content: Array[Content{ text, type}]
        
        The content of the message.
        
        text: String
        
        The text content.
        
        type: :input_text
        
        The content type. Always input_text for system messages.
        
        :input_text
      - role: :system
        
        The role of the message sender. Always system.
        
        :system
      - type: :message
        
        The type of the item. Always message.
        
        :message
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeConversationItemUserMessage
      
      A user message item in a Realtime conversation.
      - content: Array[Content{ audio, detail, image_url, 3 more}]
        
        The content of the message.
        
        audio: String
        
        Base64-encoded audio bytes (for input_audio), these will be parsed as the format specified in the session input audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
        
        detail: :auto | :low | :high
        
        The detail level of the image (for input_image). auto will default to high.
        
        :auto
        
        :low
        
        :high
        
        image_url: String
        
        Base64-encoded image bytes (for input_image) as a data URI. For example data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA.... Supported formats are PNG and JPEG.
        
        text: String
        
        The text content (for input_text).
        
        transcript: String
        
        Transcript of the audio (for input_audio). This is not sent to the model, but will be attached to the message item for reference.
        
        type: :input_text | :input_audio | :input_image
        
        The content type (input_text, input_audio, or input_image).
        
        :input_text
        
        :input_audio
        
        :input_image
      - role: :user
        
        The role of the message sender. Always user.
        
        :user
      - type: :message
        
        The type of the item. Always message.
        
        :message
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeConversationItemAssistantMessage
      
      An assistant message item in a Realtime conversation.
      - content: Array[Content{ audio, text, transcript, type}]
        
        The content of the message.
        
        audio: String
        
        Base64-encoded audio bytes, these will be parsed as the format specified in the session output audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
        
        text: String
        
        The text content.
        
        transcript: String
        
        The transcript of the audio content, this will always be present if the output type is audio.
        
        type: :output_text | :output_audio
        
        The content type, output_text or output_audio depending on the session output_modalities configuration.
        
        :output_text
        
        :output_audio
      - role: :assistant
        
        The role of the message sender. Always assistant.
        
        :assistant
      - type: :message
        
        The type of the item. Always message.
        
        :message
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeConversationItemFunctionCall
      
      A function call item in a Realtime conversation.
      - arguments: String
        
        The arguments of the function call. This is a JSON-encoded string representing the arguments passed to the function, for example {"arg1": "value1", "arg2": 42}.
      - name: String
        
        The name of the function being called.
      - type: :function_call
        
        The type of the item. Always function_call.
        
        :function_call
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - call_id: String
        
        The ID of the function call.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeConversationItemFunctionCallOutput
      
      A function call output item in a Realtime conversation.
      - call_id: String
        
        The ID of the function call this output is for.
      - output: String
        
        The output of the function call, this is free text and can contain any information or simply be empty.
      - type: :function_call_output
        
        The type of the item. Always function_call_output.
        
        :function_call_output
      - id: String
        
        The unique ID of the item. This may be provided by the client or generated by the server.
      - object: :"realtime.item"
        
        Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
        
        :"realtime.item"
      - status: :completed | :incomplete | :in_progress
        
        The status of the item. Has no effect on the conversation.
        
        :completed
        
        :incomplete
        
        :in_progress
    - class RealtimeMcpApprovalResponse
      
      A Realtime item responding to an MCP approval request.
      - id: String
        
        The unique ID of the approval response.
      - approval_request_id: String
        
        The ID of the approval request being answered.
      - approve: bool
        
        Whether the request was approved.
      - type: :mcp_approval_response
        
        The type of the item. Always mcp_approval_response.
        
        :mcp_approval_response
      - reason: String
        
        Optional reason for the decision.
    - class RealtimeMcpListTools
      
      A Realtime item listing tools available on an MCP server.
      - server_label: String
        
        The label of the MCP server.
      - tools: Array[Tool{ input_schema, name, annotations, description}]
        
        The tools available on the server.
        
        input_schema: untyped
        
        The JSON schema describing the tool's input.
        
        name: String
        
        The name of the tool.
        
        annotations: untyped
        
        Additional annotations about the tool.
        
        description: String
        
        The description of the tool.
      - type: :mcp_list_tools
        
        The type of the item. Always mcp_list_tools.
        
        :mcp_list_tools
      - id: String
        
        The unique ID of the list.
    - class RealtimeMcpToolCall
      
      A Realtime item representing an invocation of a tool on an MCP server.
      - id: String
        
        The unique ID of the tool call.
      - arguments: String
        
        A JSON string of the arguments passed to the tool.
      - name: String
        
        The name of the tool that was run.
      - server_label: String
        
        The label of the MCP server running the tool.
      - type: :mcp_call
        
        The type of the item. Always mcp_call.
        
        :mcp_call
      - approval_request_id: String
        
        The ID of an associated approval request, if any.
      - error: RealtimeMcpProtocolError | RealtimeMcpToolExecutionError | RealtimeMcphttpError
        
        The error from the tool call, if any.
        
        class RealtimeMcpProtocolError
        
        code: Integer
        
        message: String
        
        type: :protocol_error
        
        :protocol_error
        
        class RealtimeMcpToolExecutionError
        
        message: String
        
        type: :tool_execution_error
        
        :tool_execution_error
        
        class RealtimeMcphttpError
        
        code: Integer
        
        message: String
        
        type: :http_error
        
        :http_error
      - output: String
        
        The output from the tool call.
    - class RealtimeMcpApprovalRequest
      
      A Realtime item requesting human approval of a tool invocation.
      - id: String
        
        The unique ID of the approval request.
      - arguments: String
        
        A JSON string of arguments for the tool.
      - name: String
        
        The name of the tool to run.
      - server_label: String
        
        The label of the MCP server making the request.
      - type: :mcp_approval_request
        
        The type of the item. Always mcp_approval_request.
        
        :mcp_approval_request
  - output_modalities: Array[:text | :audio]
    
    The set of modalities the model used to respond, currently the only possible values are [\"audio\"], [\"text\"]. Audio output always include a text transcript. Setting the output to mode text will disable audio output from the model.
    - :text
    - :audio
  - status: :completed | :cancelled | :failed | 2 more
    
    The final status of the response (completed, cancelled, failed, or incomplete, in_progress).
    - :completed
    - :cancelled
    - :failed
    - :incomplete
    - :in_progress
  - status_details: RealtimeResponseStatus
    
    Additional details about the status.
    - error: Error{ code, type}
      
      A description of the error that caused the response to fail, populated when the status is failed.
      - code: String
        
        Error code, if any.
      - type: String
        
        The type of error.
    - reason: :turn_detected | :client_cancelled | :max_output_tokens | :content_filter
      
      The reason the Response did not complete. For a cancelled Response, one of turn_detected (the server VAD detected a new start of speech) or client_cancelled (the client sent a cancel event). For an incomplete Response, one of max_output_tokens or content_filter (the server-side safety filter activated and cut off the response).
      - :turn_detected
      - :client_cancelled
      - :max_output_tokens
      - :content_filter
    - type: :completed | :cancelled | :incomplete | :failed
      
      The type of error that caused the response to fail, corresponding with the status field (completed, cancelled, incomplete, failed).
      - :completed
      - :cancelled
      - :incomplete
      - :failed
  - usage: RealtimeResponseUsage
    
    Usage statistics for the Response, this will correspond to billing. A Realtime API session will maintain a conversation context and append new Items to the Conversation, thus output from previous turns (text and audio tokens) will become the input for later turns.
    - input_token_details: RealtimeResponseUsageInputTokenDetails
      
      Details about the input tokens used in the Response. Cached tokens are tokens from previous turns in the conversation that are included as context for the current response. Cached tokens here are counted as a subset of input tokens, meaning input tokens will include cached and uncached tokens.
      - audio_tokens: Integer
        
        The number of audio tokens used as input for the Response.
      - cached_tokens: Integer
        
        The number of cached tokens used as input for the Response.
      - cached_tokens_details: CachedTokensDetails{ audio_tokens, image_tokens, text_tokens}
        
        Details about the cached tokens used as input for the Response.
        
        audio_tokens: Integer
        
        The number of cached audio tokens used as input for the Response.
        
        image_tokens: Integer
        
        The number of cached image tokens used as input for the Response.
        
        text_tokens: Integer
        
        The number of cached text tokens used as input for the Response.
      - image_tokens: Integer
        
        The number of image tokens used as input for the Response.
      - text_tokens: Integer
        
        The number of text tokens used as input for the Response.
    - input_tokens: Integer
      
      The number of input tokens used in the Response, including text and audio tokens.
    - output_token_details: RealtimeResponseUsageOutputTokenDetails
      
      Details about the output tokens used in the Response.
      - audio_tokens: Integer
        
        The number of audio tokens used in the Response.
      - text_tokens: Integer
        
        The number of text tokens used in the Response.
    - output_tokens: Integer
      
      The number of output tokens sent in the Response, including text and audio tokens.
    - total_tokens: Integer
      
      The total number of tokens in the Response including input and output text and audio tokens.
- type: :"response.done"
  
  The event type, must be response.done.
  - :"response.done"

Response Function Call Arguments Delta Event

class ResponseFunctionCallArgumentsDeltaEvent

Returned when the model-generated function call arguments are updated.
- call_id: String
  
  The ID of the function call.
- delta: String
  
  The arguments delta as a JSON string.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the function call item.
- output_index: Integer
  
  The index of the output item in the response.
- response_id: String
  
  The ID of the response.
- type: :"response.function_call_arguments.delta"
  
  The event type, must be response.function_call_arguments.delta.
  - :"response.function_call_arguments.delta"

Response Function Call Arguments Done Event

class ResponseFunctionCallArgumentsDoneEvent

Returned when the model-generated function call arguments are done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.
- arguments: String
  
  The final arguments as a JSON string.
- call_id: String
  
  The ID of the function call.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the function call item.
- name: String
  
  The name of the function that was called.
- output_index: Integer
  
  The index of the output item in the response.
- response_id: String
  
  The ID of the response.
- type: :"response.function_call_arguments.done"
  
  The event type, must be response.function_call_arguments.done.
  - :"response.function_call_arguments.done"

Response Mcp Call Arguments Delta

class ResponseMcpCallArgumentsDelta

Returned when MCP tool call arguments are updated during response generation.
- delta: String
  
  The JSON-encoded arguments delta.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the MCP tool call item.
- output_index: Integer
  
  The index of the output item in the response.
- response_id: String
  
  The ID of the response.
- type: :"response.mcp_call_arguments.delta"
  
  The event type, must be response.mcp_call_arguments.delta.
  - :"response.mcp_call_arguments.delta"
- obfuscation: String
  
  If present, indicates the delta text was obfuscated.

Response Mcp Call Arguments Done

class ResponseMcpCallArgumentsDone

Returned when MCP tool call arguments are finalized during response generation.
- arguments: String
  
  The final JSON-encoded arguments string.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the MCP tool call item.
- output_index: Integer
  
  The index of the output item in the response.
- response_id: String
  
  The ID of the response.
- type: :"response.mcp_call_arguments.done"
  
  The event type, must be response.mcp_call_arguments.done.
  - :"response.mcp_call_arguments.done"

Response Mcp Call Completed

class ResponseMcpCallCompleted

Returned when an MCP tool call has completed successfully.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the MCP tool call item.
- output_index: Integer
  
  The index of the output item in the response.
- type: :"response.mcp_call.completed"
  
  The event type, must be response.mcp_call.completed.
  - :"response.mcp_call.completed"

Response Mcp Call Failed

class ResponseMcpCallFailed

Returned when an MCP tool call has failed.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the MCP tool call item.
- output_index: Integer
  
  The index of the output item in the response.
- type: :"response.mcp_call.failed"
  
  The event type, must be response.mcp_call.failed.
  - :"response.mcp_call.failed"

Response Mcp Call In Progress

class ResponseMcpCallInProgress

Returned when an MCP tool call has started and is in progress.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the MCP tool call item.
- output_index: Integer
  
  The index of the output item in the response.
- type: :"response.mcp_call.in_progress"
  
  The event type, must be response.mcp_call.in_progress.
  - :"response.mcp_call.in_progress"

Response Output Item Added Event

class ResponseOutputItemAddedEvent

Returned when a new Item is created during Response generation.
- event_id: String
  
  The unique ID of the server event.
- item: ConversationItem
  
  A single item within a Realtime conversation.
  - class RealtimeConversationItemSystemMessage
    
    A system message in a Realtime conversation can be used to provide additional context or instructions to the model. This is similar but distinct from the instruction prompt provided at the start of a conversation, as system messages can be added at any point in the conversation. For major changes to the conversation's behavior, use instructions, but for smaller updates (e.g. "the user is now asking about a different topic"), use system messages.
    - content: Array[Content{ text, type}]
      
      The content of the message.
      - text: String
        
        The text content.
      - type: :input_text
        
        The content type. Always input_text for system messages.
        
        :input_text
    - role: :system
      
      The role of the message sender. Always system.
      - :system
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemUserMessage
    
    A user message item in a Realtime conversation.
    - content: Array[Content{ audio, detail, image_url, 3 more}]
      
      The content of the message.
      - audio: String
        
        Base64-encoded audio bytes (for input_audio), these will be parsed as the format specified in the session input audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
      - detail: :auto | :low | :high
        
        The detail level of the image (for input_image). auto will default to high.
        
        :auto
        
        :low
        
        :high
      - image_url: String
        
        Base64-encoded image bytes (for input_image) as a data URI. For example data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA.... Supported formats are PNG and JPEG.
      - text: String
        
        The text content (for input_text).
      - transcript: String
        
        Transcript of the audio (for input_audio). This is not sent to the model, but will be attached to the message item for reference.
      - type: :input_text | :input_audio | :input_image
        
        The content type (input_text, input_audio, or input_image).
        
        :input_text
        
        :input_audio
        
        :input_image
    - role: :user
      
      The role of the message sender. Always user.
      - :user
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemAssistantMessage
    
    An assistant message item in a Realtime conversation.
    - content: Array[Content{ audio, text, transcript, type}]
      
      The content of the message.
      - audio: String
        
        Base64-encoded audio bytes, these will be parsed as the format specified in the session output audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
      - text: String
        
        The text content.
      - transcript: String
        
        The transcript of the audio content, this will always be present if the output type is audio.
      - type: :output_text | :output_audio
        
        The content type, output_text or output_audio depending on the session output_modalities configuration.
        
        :output_text
        
        :output_audio
    - role: :assistant
      
      The role of the message sender. Always assistant.
      - :assistant
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemFunctionCall
    
    A function call item in a Realtime conversation.
    - arguments: String
      
      The arguments of the function call. This is a JSON-encoded string representing the arguments passed to the function, for example {"arg1": "value1", "arg2": 42}.
    - name: String
      
      The name of the function being called.
    - type: :function_call
      
      The type of the item. Always function_call.
      - :function_call
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - call_id: String
      
      The ID of the function call.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemFunctionCallOutput
    
    A function call output item in a Realtime conversation.
    - call_id: String
      
      The ID of the function call this output is for.
    - output: String
      
      The output of the function call, this is free text and can contain any information or simply be empty.
    - type: :function_call_output
      
      The type of the item. Always function_call_output.
      - :function_call_output
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeMcpApprovalResponse
    
    A Realtime item responding to an MCP approval request.
    - id: String
      
      The unique ID of the approval response.
    - approval_request_id: String
      
      The ID of the approval request being answered.
    - approve: bool
      
      Whether the request was approved.
    - type: :mcp_approval_response
      
      The type of the item. Always mcp_approval_response.
      - :mcp_approval_response
    - reason: String
      
      Optional reason for the decision.
  - class RealtimeMcpListTools
    
    A Realtime item listing tools available on an MCP server.
    - server_label: String
      
      The label of the MCP server.
    - tools: Array[Tool{ input_schema, name, annotations, description}]
      
      The tools available on the server.
      - input_schema: untyped
        
        The JSON schema describing the tool's input.
      - name: String
        
        The name of the tool.
      - annotations: untyped
        
        Additional annotations about the tool.
      - description: String
        
        The description of the tool.
    - type: :mcp_list_tools
      
      The type of the item. Always mcp_list_tools.
      - :mcp_list_tools
    - id: String
      
      The unique ID of the list.
  - class RealtimeMcpToolCall
    
    A Realtime item representing an invocation of a tool on an MCP server.
    - id: String
      
      The unique ID of the tool call.
    - arguments: String
      
      A JSON string of the arguments passed to the tool.
    - name: String
      
      The name of the tool that was run.
    - server_label: String
      
      The label of the MCP server running the tool.
    - type: :mcp_call
      
      The type of the item. Always mcp_call.
      - :mcp_call
    - approval_request_id: String
      
      The ID of an associated approval request, if any.
    - error: RealtimeMcpProtocolError | RealtimeMcpToolExecutionError | RealtimeMcphttpError
      
      The error from the tool call, if any.
      - class RealtimeMcpProtocolError
        
        code: Integer
        
        message: String
        
        type: :protocol_error
        
        :protocol_error
      - class RealtimeMcpToolExecutionError
        
        message: String
        
        type: :tool_execution_error
        
        :tool_execution_error
      - class RealtimeMcphttpError
        
        code: Integer
        
        message: String
        
        type: :http_error
        
        :http_error
    - output: String
      
      The output from the tool call.
  - class RealtimeMcpApprovalRequest
    
    A Realtime item requesting human approval of a tool invocation.
    - id: String
      
      The unique ID of the approval request.
    - arguments: String
      
      A JSON string of arguments for the tool.
    - name: String
      
      The name of the tool to run.
    - server_label: String
      
      The label of the MCP server making the request.
    - type: :mcp_approval_request
      
      The type of the item. Always mcp_approval_request.
      - :mcp_approval_request
- output_index: Integer
  
  The index of the output item in the Response.
- response_id: String
  
  The ID of the Response to which the item belongs.
- type: :"response.output_item.added"
  
  The event type, must be response.output_item.added.
  - :"response.output_item.added"

Response Output Item Done Event

class ResponseOutputItemDoneEvent

Returned when an Item is done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.
- event_id: String
  
  The unique ID of the server event.
- item: ConversationItem
  
  A single item within a Realtime conversation.
  - class RealtimeConversationItemSystemMessage
    
    A system message in a Realtime conversation can be used to provide additional context or instructions to the model. This is similar but distinct from the instruction prompt provided at the start of a conversation, as system messages can be added at any point in the conversation. For major changes to the conversation's behavior, use instructions, but for smaller updates (e.g. "the user is now asking about a different topic"), use system messages.
    - content: Array[Content{ text, type}]
      
      The content of the message.
      - text: String
        
        The text content.
      - type: :input_text
        
        The content type. Always input_text for system messages.
        
        :input_text
    - role: :system
      
      The role of the message sender. Always system.
      - :system
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemUserMessage
    
    A user message item in a Realtime conversation.
    - content: Array[Content{ audio, detail, image_url, 3 more}]
      
      The content of the message.
      - audio: String
        
        Base64-encoded audio bytes (for input_audio), these will be parsed as the format specified in the session input audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
      - detail: :auto | :low | :high
        
        The detail level of the image (for input_image). auto will default to high.
        
        :auto
        
        :low
        
        :high
      - image_url: String
        
        Base64-encoded image bytes (for input_image) as a data URI. For example data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA.... Supported formats are PNG and JPEG.
      - text: String
        
        The text content (for input_text).
      - transcript: String
        
        Transcript of the audio (for input_audio). This is not sent to the model, but will be attached to the message item for reference.
      - type: :input_text | :input_audio | :input_image
        
        The content type (input_text, input_audio, or input_image).
        
        :input_text
        
        :input_audio
        
        :input_image
    - role: :user
      
      The role of the message sender. Always user.
      - :user
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemAssistantMessage
    
    An assistant message item in a Realtime conversation.
    - content: Array[Content{ audio, text, transcript, type}]
      
      The content of the message.
      - audio: String
        
        Base64-encoded audio bytes, these will be parsed as the format specified in the session output audio type configuration. This defaults to PCM 16-bit 24kHz mono if not specified.
      - text: String
        
        The text content.
      - transcript: String
        
        The transcript of the audio content, this will always be present if the output type is audio.
      - type: :output_text | :output_audio
        
        The content type, output_text or output_audio depending on the session output_modalities configuration.
        
        :output_text
        
        :output_audio
    - role: :assistant
      
      The role of the message sender. Always assistant.
      - :assistant
    - type: :message
      
      The type of the item. Always message.
      - :message
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemFunctionCall
    
    A function call item in a Realtime conversation.
    - arguments: String
      
      The arguments of the function call. This is a JSON-encoded string representing the arguments passed to the function, for example {"arg1": "value1", "arg2": 42}.
    - name: String
      
      The name of the function being called.
    - type: :function_call
      
      The type of the item. Always function_call.
      - :function_call
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - call_id: String
      
      The ID of the function call.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeConversationItemFunctionCallOutput
    
    A function call output item in a Realtime conversation.
    - call_id: String
      
      The ID of the function call this output is for.
    - output: String
      
      The output of the function call, this is free text and can contain any information or simply be empty.
    - type: :function_call_output
      
      The type of the item. Always function_call_output.
      - :function_call_output
    - id: String
      
      The unique ID of the item. This may be provided by the client or generated by the server.
    - object: :"realtime.item"
      
      Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
      - :"realtime.item"
    - status: :completed | :incomplete | :in_progress
      
      The status of the item. Has no effect on the conversation.
      - :completed
      - :incomplete
      - :in_progress
  - class RealtimeMcpApprovalResponse
    
    A Realtime item responding to an MCP approval request.
    - id: String
      
      The unique ID of the approval response.
    - approval_request_id: String
      
      The ID of the approval request being answered.
    - approve: bool
      
      Whether the request was approved.
    - type: :mcp_approval_response
      
      The type of the item. Always mcp_approval_response.
      - :mcp_approval_response
    - reason: String
      
      Optional reason for the decision.
  - class RealtimeMcpListTools
    
    A Realtime item listing tools available on an MCP server.
    - server_label: String
      
      The label of the MCP server.
    - tools: Array[Tool{ input_schema, name, annotations, description}]
      
      The tools available on the server.
      - input_schema: untyped
        
        The JSON schema describing the tool's input.
      - name: String
        
        The name of the tool.
      - annotations: untyped
        
        Additional annotations about the tool.
      - description: String
        
        The description of the tool.
    - type: :mcp_list_tools
      
      The type of the item. Always mcp_list_tools.
      - :mcp_list_tools
    - id: String
      
      The unique ID of the list.
  - class RealtimeMcpToolCall
    
    A Realtime item representing an invocation of a tool on an MCP server.
    - id: String
      
      The unique ID of the tool call.
    - arguments: String
      
      A JSON string of the arguments passed to the tool.
    - name: String
      
      The name of the tool that was run.
    - server_label: String
      
      The label of the MCP server running the tool.
    - type: :mcp_call
      
      The type of the item. Always mcp_call.
      - :mcp_call
    - approval_request_id: String
      
      The ID of an associated approval request, if any.
    - error: RealtimeMcpProtocolError | RealtimeMcpToolExecutionError | RealtimeMcphttpError
      
      The error from the tool call, if any.
      - class RealtimeMcpProtocolError
        
        code: Integer
        
        message: String
        
        type: :protocol_error
        
        :protocol_error
      - class RealtimeMcpToolExecutionError
        
        message: String
        
        type: :tool_execution_error
        
        :tool_execution_error
      - class RealtimeMcphttpError
        
        code: Integer
        
        message: String
        
        type: :http_error
        
        :http_error
    - output: String
      
      The output from the tool call.
  - class RealtimeMcpApprovalRequest
    
    A Realtime item requesting human approval of a tool invocation.
    - id: String
      
      The unique ID of the approval request.
    - arguments: String
      
      A JSON string of arguments for the tool.
    - name: String
      
      The name of the tool to run.
    - server_label: String
      
      The label of the MCP server making the request.
    - type: :mcp_approval_request
      
      The type of the item. Always mcp_approval_request.
      - :mcp_approval_request
- output_index: Integer
  
  The index of the output item in the Response.
- response_id: String
  
  The ID of the Response to which the item belongs.
- type: :"response.output_item.done"
  
  The event type, must be response.output_item.done.
  - :"response.output_item.done"

Response Text Delta Event

class ResponseTextDeltaEvent

Returned when the text value of an "output_text" content part is updated.
- content_index: Integer
  
  The index of the content part in the item's content array.
- delta: String
  
  The text delta.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the item.
- output_index: Integer
  
  The index of the output item in the response.
- response_id: String
  
  The ID of the response.
- type: :"response.output_text.delta"
  
  The event type, must be response.output_text.delta.
  - :"response.output_text.delta"

Response Text Done Event

class ResponseTextDoneEvent

Returned when the text value of an "output_text" content part is done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.
- content_index: Integer
  
  The index of the content part in the item's content array.
- event_id: String
  
  The unique ID of the server event.
- item_id: String
  
  The ID of the item.
- output_index: Integer
  
  The index of the output item in the response.
- response_id: String
  
  The ID of the response.
- text: String
  
  The final text content.
- type: :"response.output_text.done"
  
  The event type, must be response.output_text.done.
  - :"response.output_text.done"

Session Created Event

class SessionCreatedEvent

Returned when a Session is created. Emitted automatically when a new connection is established as the first server event. This event will contain the default Session configuration.
- event_id: String
  
  The unique ID of the server event.
- session: RealtimeSessionCreateRequest | RealtimeTranscriptionSessionCreateRequest
  
  The session configuration.
  - class RealtimeSessionCreateRequest
    
    Realtime session object configuration.
    - type: :realtime
      
      The type of session to create. Always realtime for the Realtime API.
      - :realtime
    - audio: RealtimeAudioConfig
      
      Configuration for input and output audio.
      - input: RealtimeAudioConfigInput
        
        format_: RealtimeAudioFormats
        
        The format of the input audio.
        
        class AudioPCM
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
        
        type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
        
        class AudioPCMU
        
        The G.711 μ-law format.
        
        type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
        
        class AudioPCMA
        
        The G.711 A-law format.
        
        type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
        
        noise_reduction: NoiseReduction{ type}
        
        Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
        
        transcription: AudioTranscription
        
        Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
        
        delay: :minimal | :low | :medium | 2 more
        
        Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
        
        :minimal
        
        :low
        
        :medium
        
        :high
        
        :xhigh
        
        language: String
        
        The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
        
        model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        String = String
        
        Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        :"whisper-1"
        
        :"gpt-4o-mini-transcribe"
        
        :"gpt-4o-mini-transcribe-2025-12-15"
        
        :"gpt-4o-transcribe"
        
        :"gpt-4o-transcribe-diarize"
        
        :"gpt-realtime-whisper"
        
        prompt: String
        
        An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
        
        turn_detection: RealtimeAudioInputTurnDetection
        
        Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
        
        Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
        
        Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
        
        For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
        
        class ServerVad
        
        Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
        
        type: :server_vad
        
        Type of turn detection, server_vad to turn on simple Server VAD.
        
        :server_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        idle_timeout_ms: Integer
        
        Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
        
        The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
        
        An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        prefix_padding_ms: Integer
        
        Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
        
        silence_duration_ms: Integer
        
        Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
        
        threshold: Float
        
        Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
        
        class SemanticVad
        
        Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
        
        type: :semantic_vad
        
        Type of turn detection, semantic_vad to turn on Semantic VAD.
        
        :semantic_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs.
        
        eagerness: :low | :medium | :high | :auto
        
        Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
        
        :low
        
        :medium
        
        :high
        
        :auto
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.
      - output: RealtimeAudioConfigOutput
        
        format_: RealtimeAudioFormats
        
        The format of the output audio.
        
        speed: Float
        
        The speed of the model's spoken response as a multiple of the original speed. 1.0 is the default speed. 0.25 is the minimum speed. 1.5 is the maximum speed. This value can only be changed in between model turns, not while a response is in progress.
        
        This parameter is a post-processing adjustment to the audio after it is generated, it's also possible to prompt the model to speak faster or slower.
        
        voice: String | :alloy | :ash | :ballad | 7 more | ID{ id}
        
        The voice the model uses to respond. Supported built-in voices are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. You may also provide a custom voice object with an id, for example { "id": "voice_1234" }. Voice cannot be changed during the session once the model has responded with audio at least once. We recommend marin and cedar for best quality.
        
        String = String
        
        Voice = :alloy | :ash | :ballad | 7 more
        
        :alloy
        
        :ash
        
        :ballad
        
        :coral
        
        :echo
        
        :sage
        
        :shimmer
        
        :verse
        
        :marin
        
        :cedar
        
        class ID
        
        Custom voice reference.
        
        id: String
        
        The custom voice ID, e.g. voice_1234.
    - include: Array[:"item.input_audio_transcription.logprobs"]
      
      Additional fields to include in server outputs.
      
      item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
      - :"item.input_audio_transcription.logprobs"
    - instructions: String
      
      The default system instructions (i.e. system message) prepended to model calls. This field allows the client to guide the model on desired responses. The model can be instructed on response content and format, (e.g. "be extremely succinct", "act friendly", "here are examples of good responses") and on audio behavior (e.g. "talk quickly", "inject emotion into your voice", "laugh frequently"). The instructions are not guaranteed to be followed by the model, but they provide guidance to the model on the desired behavior.
      
      Note that the server sets default instructions which will be used if this field is not set and are visible in the session.created event at the start of the session.
    - max_output_tokens: Integer | :inf
      
      Maximum number of output tokens for a single assistant response, inclusive of tool calls. Provide an integer between 1 and 4096 to limit output tokens, or inf for the maximum available tokens for a given model. Defaults to inf.
      - Integer = Integer
      - MaxOutputTokens = :inf
        
        :inf
    - model: String | :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
      
      The Realtime model used for this session.
      - String = String
      - Model = :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
        
        The Realtime model used for this session.
        
        :"gpt-realtime"
        
        :"gpt-realtime-1.5"
        
        :"gpt-realtime-2"
        
        :"gpt-realtime-2025-08-28"
        
        :"gpt-4o-realtime-preview"
        
        :"gpt-4o-realtime-preview-2024-10-01"
        
        :"gpt-4o-realtime-preview-2024-12-17"
        
        :"gpt-4o-realtime-preview-2025-06-03"
        
        :"gpt-4o-mini-realtime-preview"
        
        :"gpt-4o-mini-realtime-preview-2024-12-17"
        
        :"gpt-realtime-mini"
        
        :"gpt-realtime-mini-2025-10-06"
        
        :"gpt-realtime-mini-2025-12-15"
        
        :"gpt-audio-1.5"
        
        :"gpt-audio-mini"
        
        :"gpt-audio-mini-2025-10-06"
        
        :"gpt-audio-mini-2025-12-15"
    - output_modalities: Array[:text | :audio]
      
      The set of modalities the model can respond with. It defaults to ["audio"], indicating that the model will respond with audio plus a transcript. ["text"] can be used to make the model respond with text only. It is not possible to request both text and audio at the same time.
      - :text
      - :audio
    - parallel_tool_calls: bool
      
      Whether the model may call multiple tools in parallel. Only supported by reasoning Realtime models such as gpt-realtime-2.
    - prompt: ResponsePrompt
      
      Reference to a prompt template and its variables. Learn more.
      - id: String
        
        The unique identifier of the prompt template to use.
      - variables: Hash[Symbol, String | ResponseInputText | ResponseInputImage | ResponseInputFile]
        
        Optional map of values to substitute in for variables in your prompt. The substitution values can either be strings, or other Response input types like images or files.
        
        String = String
        
        class ResponseInputText
        
        A text input to the model.
        
        text: String
        
        The text input to the model.
        
        type: :input_text
        
        The type of the input item. Always input_text.
        
        :input_text
        
        class ResponseInputImage
        
        An image input to the model. Learn about image inputs.
        
        detail: :low | :high | :auto | :original
        
        The detail level of the image to be sent to the model. One of high, low, auto, or original. Defaults to auto.
        
        :low
        
        :high
        
        :auto
        
        :original
        
        type: :input_image
        
        The type of the input item. Always input_image.
        
        :input_image
        
        file_id: String
        
        The ID of the file to be sent to the model.
        
        image_url: String
        
        The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.
        
        class ResponseInputFile
        
        A file input to the model.
        
        type: :input_file
        
        The type of the input item. Always input_file.
        
        :input_file
        
        detail: :low | :high
        
        The detail level of the file to be sent to the model. Use low for the default rendering behavior, or high to render the file at higher quality. Defaults to low.
        
        :low
        
        :high
        
        file_data: String
        
        The content of the file to be sent to the model.
        
        file_id: String
        
        The ID of the file to be sent to the model.
        
        file_url: String
        
        The URL of the file to be sent to the model.
        
        filename: String
        
        The name of the file to be sent to the model.
      - version: String
        
        Optional version of the prompt template.
    - reasoning: RealtimeReasoning
      
      Configuration for reasoning-capable Realtime models such as gpt-realtime-2.
      - effort: RealtimeReasoningEffort
        
        Constrains effort on reasoning for reasoning-capable Realtime models such as gpt-realtime-2.
        
        :minimal
        
        :low
        
        :medium
        
        :high
        
        :xhigh
    - tool_choice: RealtimeToolChoiceConfig
      
      How the model chooses tools. Provide one of the string modes or force a specific function/MCP tool.
      - ToolChoiceOptions = :none | :auto | :required
        
        Controls which (if any) tool is called by the model.
        
        none means the model will not call any tool and instead generates a message.
        
        auto means the model can pick between generating a message or calling one or more tools.
        
        required means the model must call one or more tools.
        
        :none
        
        :auto
        
        :required
      - class ToolChoiceFunction
        
        Use this option to force the model to call a specific function.
        
        name: String
        
        The name of the function to call.
        
        type: :function
        
        For function calling, the type is always function.
        
        :function
      - class ToolChoiceMcp
        
        Use this option to force the model to call a specific tool on a remote MCP server.
        
        server_label: String
        
        The label of the MCP server to use.
        
        type: :mcp
        
        For MCP tools, the type is always mcp.
        
        :mcp
        
        name: String
        
        The name of the tool to call on the server.
    - tools: RealtimeToolsConfig
      
      Tools available to the model.
      - class RealtimeFunctionTool
        
        description: String
        
        The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything).
        
        name: String
        
        The name of the function.
        
        parameters: untyped
        
        Parameters of the function in JSON Schema.
        
        type: :function
        
        The type of the tool, i.e. function.
        
        :function
      - class Mcp
        
        Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.
        
        server_label: String
        
        A label for this MCP server, used to identify it in tool calls.
        
        type: :mcp
        
        The type of the MCP tool. Always mcp.
        
        :mcp
        
        allowed_tools: Array[String] | McpToolFilter{ read_only, tool_names}
        
        List of allowed tool names or a filter object.
        
        McpAllowedTools = Array[String]
        
        A string array of allowed tool names
        
        class McpToolFilter
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        authorization: String
        
        An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.
        
        connector_id: :connector_dropbox | :connector_gmail | :connector_googlecalendar | 5 more
        
        Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here.
        
        Currently supported connector_id values are:
        
        Dropbox: connector_dropbox
        
        Gmail: connector_gmail
        
        Google Calendar: connector_googlecalendar
        
        Google Drive: connector_googledrive
        
        Microsoft Teams: connector_microsoftteams
        
        Outlook Calendar: connector_outlookcalendar
        
        Outlook Email: connector_outlookemail
        
        SharePoint: connector_sharepoint
        
        :connector_dropbox
        
        :connector_gmail
        
        :connector_googlecalendar
        
        :connector_googledrive
        
        :connector_microsoftteams
        
        :connector_outlookcalendar
        
        :connector_outlookemail
        
        :connector_sharepoint
        
        defer_loading: bool
        
        Whether this MCP tool is deferred and discovered via tool search.
        
        headers: Hash[Symbol, String]
        
        Optional HTTP headers to send to the MCP server. Use for authentication or other purposes.
        
        require_approval: McpToolApprovalFilter{ always, never} | :always | :never
        
        Specify which of the MCP server's tools require approval.
        
        class McpToolApprovalFilter
        
        Specify which of the MCP server's tools require approval. Can be always, never, or a filter object associated with tools that require approval.
        
        always: Always{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        never: Never{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        McpToolApprovalSetting = :always | :never
        
        Specify a single approval policy for all tools. One of always or never. When set to always, all tools will require approval. When set to never, all tools will not require approval.
        
        :always
        
        :never
        
        server_description: String
        
        Optional description of the MCP server, used to provide more context.
        
        server_url: String
        
        The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided.
        
        tunnel_id: String
        
        The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided.
    - tracing: RealtimeTracingConfig
      
      Realtime API can write session traces to the Traces Dashboard. Set to null to disable tracing. Once tracing is enabled for a session, the configuration cannot be modified.
      
      auto will create a trace for the session with default values for the workflow name, group id, and metadata.
      - RealtimeTracingConfig = :auto
        
        Enables tracing and sets default values for tracing configuration options. Always auto.
        
        :auto
      - class TracingConfiguration
        
        Granular configuration for tracing.
        
        group_id: String
        
        The group id to attach to this trace to enable filtering and grouping in the Traces Dashboard.
        
        metadata: untyped
        
        The arbitrary metadata to attach to this trace to enable filtering in the Traces Dashboard.
        
        workflow_name: String
        
        The name of the workflow to attach to this trace. This is used to name the trace in the Traces Dashboard.
    - truncation: RealtimeTruncation
      
      When the number of tokens in a conversation exceeds the model's input token limit, the conversation be truncated, meaning messages (starting from the oldest) will not be included in the model's context. A 32k context model with 4,096 max output tokens can only include 28,224 tokens in the context before truncation occurs.
      
      Clients can configure truncation behavior to truncate with a lower max token limit, which is an effective way to control token usage and cost.
      
      Truncation will reduce the number of cached tokens on the next turn (busting the cache), since messages are dropped from the beginning of the context. However, clients can also configure truncation to retain messages up to a fraction of the maximum context size, which will reduce the need for future truncations and thus improve the cache rate.
      
      Truncation can be disabled entirely, which means the server will never truncate but would instead return an error if the conversation exceeds the model's input token limit.
      - RealtimeTruncationStrategy = :auto | :disabled
        
        The truncation strategy to use for the session. auto is the default truncation strategy. disabled will disable truncation and emit errors when the conversation exceeds the input token limit.
        
        :auto
        
        :disabled
      - class RealtimeTruncationRetentionRatio
        
        Retain a fraction of the conversation tokens when the conversation exceeds the input token limit. This allows you to amortize truncations across multiple turns, which can help improve cached token usage.
        
        retention_ratio: Float
        
        Fraction of post-instruction conversation tokens to retain (0.0 - 1.0) when the conversation exceeds the input token limit. Setting this to 0.8 means that messages will be dropped until 80% of the maximum allowed tokens are used. This helps reduce the frequency of truncations and improve cache rates.
        
        type: :retention_ratio
        
        Use retention ratio truncation.
        
        :retention_ratio
        
        token_limits: TokenLimits{ post_instructions}
        
        Optional custom token limits for this truncation strategy. If not provided, the model's default token limits will be used.
        
        post_instructions: Integer
        
        Maximum tokens allowed in the conversation after instructions (which including tool definitions). For example, setting this to 5,000 would mean that truncation would occur when the conversation exceeds 5,000 tokens after instructions. This cannot be higher than the model's context window size minus the maximum output tokens.
  - class RealtimeTranscriptionSessionCreateRequest
    
    Realtime transcription session object configuration.
    - type: :transcription
      
      The type of session to create. Always transcription for transcription sessions.
      - :transcription
    - audio: RealtimeTranscriptionSessionAudio
      
      Configuration for input and output audio.
      - input: RealtimeTranscriptionSessionAudioInput
        
        format_: RealtimeAudioFormats
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        noise_reduction: NoiseReduction{ type}
        
        Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        transcription: AudioTranscription
        
        Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
        
        turn_detection: RealtimeTranscriptionSessionAudioInputTurnDetection
        
        Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
        
        Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
        
        Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
        
        For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
        
        class ServerVad
        
        Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
        
        type: :server_vad
        
        Type of turn detection, server_vad to turn on simple Server VAD.
        
        :server_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        idle_timeout_ms: Integer
        
        Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
        
        The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
        
        An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        prefix_padding_ms: Integer
        
        Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
        
        silence_duration_ms: Integer
        
        Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
        
        threshold: Float
        
        Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
        
        class SemanticVad
        
        Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
        
        type: :semantic_vad
        
        Type of turn detection, semantic_vad to turn on Semantic VAD.
        
        :semantic_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs.
        
        eagerness: :low | :medium | :high | :auto
        
        Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
        
        :low
        
        :medium
        
        :high
        
        :auto
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.
    - include: Array[:"item.input_audio_transcription.logprobs"]
      
      Additional fields to include in server outputs.
      
      item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
      - :"item.input_audio_transcription.logprobs"
- type: :"session.created"
  
  The event type, must be session.created.
  - :"session.created"

Session Update Event

class SessionUpdateEvent

Send this event to update the session’s configuration. The client may send this event at any time to update any field except for voice and model. voice can be updated only if there have been no other audio outputs yet.

When the server receives a session.update, it will respond with a session.updated event showing the full, effective configuration. Only the fields that are present in the session.update are updated. To clear a field like instructions, pass an empty string. To clear a field like tools, pass an empty array. To clear a field like turn_detection, pass null.
- session: RealtimeSessionCreateRequest | RealtimeTranscriptionSessionCreateRequest
  
  Update the Realtime session. Choose either a realtime session or a transcription session.
  - class RealtimeSessionCreateRequest
    
    Realtime session object configuration.
    - type: :realtime
      
      The type of session to create. Always realtime for the Realtime API.
      - :realtime
    - audio: RealtimeAudioConfig
      
      Configuration for input and output audio.
      - input: RealtimeAudioConfigInput
        
        format_: RealtimeAudioFormats
        
        The format of the input audio.
        
        class AudioPCM
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
        
        type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
        
        class AudioPCMU
        
        The G.711 μ-law format.
        
        type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
        
        class AudioPCMA
        
        The G.711 A-law format.
        
        type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
        
        noise_reduction: NoiseReduction{ type}
        
        Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
        
        transcription: AudioTranscription
        
        Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
        
        delay: :minimal | :low | :medium | 2 more
        
        Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
        
        :minimal
        
        :low
        
        :medium
        
        :high
        
        :xhigh
        
        language: String
        
        The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
        
        model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        String = String
        
        Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        :"whisper-1"
        
        :"gpt-4o-mini-transcribe"
        
        :"gpt-4o-mini-transcribe-2025-12-15"
        
        :"gpt-4o-transcribe"
        
        :"gpt-4o-transcribe-diarize"
        
        :"gpt-realtime-whisper"
        
        prompt: String
        
        An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
        
        turn_detection: RealtimeAudioInputTurnDetection
        
        Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
        
        Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
        
        Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
        
        For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
        
        class ServerVad
        
        Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
        
        type: :server_vad
        
        Type of turn detection, server_vad to turn on simple Server VAD.
        
        :server_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        idle_timeout_ms: Integer
        
        Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
        
        The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
        
        An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        prefix_padding_ms: Integer
        
        Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
        
        silence_duration_ms: Integer
        
        Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
        
        threshold: Float
        
        Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
        
        class SemanticVad
        
        Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
        
        type: :semantic_vad
        
        Type of turn detection, semantic_vad to turn on Semantic VAD.
        
        :semantic_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs.
        
        eagerness: :low | :medium | :high | :auto
        
        Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
        
        :low
        
        :medium
        
        :high
        
        :auto
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.
      - output: RealtimeAudioConfigOutput
        
        format_: RealtimeAudioFormats
        
        The format of the output audio.
        
        speed: Float
        
        The speed of the model's spoken response as a multiple of the original speed. 1.0 is the default speed. 0.25 is the minimum speed. 1.5 is the maximum speed. This value can only be changed in between model turns, not while a response is in progress.
        
        This parameter is a post-processing adjustment to the audio after it is generated, it's also possible to prompt the model to speak faster or slower.
        
        voice: String | :alloy | :ash | :ballad | 7 more | ID{ id}
        
        The voice the model uses to respond. Supported built-in voices are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. You may also provide a custom voice object with an id, for example { "id": "voice_1234" }. Voice cannot be changed during the session once the model has responded with audio at least once. We recommend marin and cedar for best quality.
        
        String = String
        
        Voice = :alloy | :ash | :ballad | 7 more
        
        :alloy
        
        :ash
        
        :ballad
        
        :coral
        
        :echo
        
        :sage
        
        :shimmer
        
        :verse
        
        :marin
        
        :cedar
        
        class ID
        
        Custom voice reference.
        
        id: String
        
        The custom voice ID, e.g. voice_1234.
    - include: Array[:"item.input_audio_transcription.logprobs"]
      
      Additional fields to include in server outputs.
      
      item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
      - :"item.input_audio_transcription.logprobs"
    - instructions: String
      
      The default system instructions (i.e. system message) prepended to model calls. This field allows the client to guide the model on desired responses. The model can be instructed on response content and format, (e.g. "be extremely succinct", "act friendly", "here are examples of good responses") and on audio behavior (e.g. "talk quickly", "inject emotion into your voice", "laugh frequently"). The instructions are not guaranteed to be followed by the model, but they provide guidance to the model on the desired behavior.
      
      Note that the server sets default instructions which will be used if this field is not set and are visible in the session.created event at the start of the session.
    - max_output_tokens: Integer | :inf
      
      Maximum number of output tokens for a single assistant response, inclusive of tool calls. Provide an integer between 1 and 4096 to limit output tokens, or inf for the maximum available tokens for a given model. Defaults to inf.
      - Integer = Integer
      - MaxOutputTokens = :inf
        
        :inf
    - model: String | :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
      
      The Realtime model used for this session.
      - String = String
      - Model = :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
        
        The Realtime model used for this session.
        
        :"gpt-realtime"
        
        :"gpt-realtime-1.5"
        
        :"gpt-realtime-2"
        
        :"gpt-realtime-2025-08-28"
        
        :"gpt-4o-realtime-preview"
        
        :"gpt-4o-realtime-preview-2024-10-01"
        
        :"gpt-4o-realtime-preview-2024-12-17"
        
        :"gpt-4o-realtime-preview-2025-06-03"
        
        :"gpt-4o-mini-realtime-preview"
        
        :"gpt-4o-mini-realtime-preview-2024-12-17"
        
        :"gpt-realtime-mini"
        
        :"gpt-realtime-mini-2025-10-06"
        
        :"gpt-realtime-mini-2025-12-15"
        
        :"gpt-audio-1.5"
        
        :"gpt-audio-mini"
        
        :"gpt-audio-mini-2025-10-06"
        
        :"gpt-audio-mini-2025-12-15"
    - output_modalities: Array[:text | :audio]
      
      The set of modalities the model can respond with. It defaults to ["audio"], indicating that the model will respond with audio plus a transcript. ["text"] can be used to make the model respond with text only. It is not possible to request both text and audio at the same time.
      - :text
      - :audio
    - parallel_tool_calls: bool
      
      Whether the model may call multiple tools in parallel. Only supported by reasoning Realtime models such as gpt-realtime-2.
    - prompt: ResponsePrompt
      
      Reference to a prompt template and its variables. Learn more.
      - id: String
        
        The unique identifier of the prompt template to use.
      - variables: Hash[Symbol, String | ResponseInputText | ResponseInputImage | ResponseInputFile]
        
        Optional map of values to substitute in for variables in your prompt. The substitution values can either be strings, or other Response input types like images or files.
        
        String = String
        
        class ResponseInputText
        
        A text input to the model.
        
        text: String
        
        The text input to the model.
        
        type: :input_text
        
        The type of the input item. Always input_text.
        
        :input_text
        
        class ResponseInputImage
        
        An image input to the model. Learn about image inputs.
        
        detail: :low | :high | :auto | :original
        
        The detail level of the image to be sent to the model. One of high, low, auto, or original. Defaults to auto.
        
        :low
        
        :high
        
        :auto
        
        :original
        
        type: :input_image
        
        The type of the input item. Always input_image.
        
        :input_image
        
        file_id: String
        
        The ID of the file to be sent to the model.
        
        image_url: String
        
        The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.
        
        class ResponseInputFile
        
        A file input to the model.
        
        type: :input_file
        
        The type of the input item. Always input_file.
        
        :input_file
        
        detail: :low | :high
        
        The detail level of the file to be sent to the model. Use low for the default rendering behavior, or high to render the file at higher quality. Defaults to low.
        
        :low
        
        :high
        
        file_data: String
        
        The content of the file to be sent to the model.
        
        file_id: String
        
        The ID of the file to be sent to the model.
        
        file_url: String
        
        The URL of the file to be sent to the model.
        
        filename: String
        
        The name of the file to be sent to the model.
      - version: String
        
        Optional version of the prompt template.
    - reasoning: RealtimeReasoning
      
      Configuration for reasoning-capable Realtime models such as gpt-realtime-2.
      - effort: RealtimeReasoningEffort
        
        Constrains effort on reasoning for reasoning-capable Realtime models such as gpt-realtime-2.
        
        :minimal
        
        :low
        
        :medium
        
        :high
        
        :xhigh
    - tool_choice: RealtimeToolChoiceConfig
      
      How the model chooses tools. Provide one of the string modes or force a specific function/MCP tool.
      - ToolChoiceOptions = :none | :auto | :required
        
        Controls which (if any) tool is called by the model.
        
        none means the model will not call any tool and instead generates a message.
        
        auto means the model can pick between generating a message or calling one or more tools.
        
        required means the model must call one or more tools.
        
        :none
        
        :auto
        
        :required
      - class ToolChoiceFunction
        
        Use this option to force the model to call a specific function.
        
        name: String
        
        The name of the function to call.
        
        type: :function
        
        For function calling, the type is always function.
        
        :function
      - class ToolChoiceMcp
        
        Use this option to force the model to call a specific tool on a remote MCP server.
        
        server_label: String
        
        The label of the MCP server to use.
        
        type: :mcp
        
        For MCP tools, the type is always mcp.
        
        :mcp
        
        name: String
        
        The name of the tool to call on the server.
    - tools: RealtimeToolsConfig
      
      Tools available to the model.
      - class RealtimeFunctionTool
        
        description: String
        
        The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything).
        
        name: String
        
        The name of the function.
        
        parameters: untyped
        
        Parameters of the function in JSON Schema.
        
        type: :function
        
        The type of the tool, i.e. function.
        
        :function
      - class Mcp
        
        Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.
        
        server_label: String
        
        A label for this MCP server, used to identify it in tool calls.
        
        type: :mcp
        
        The type of the MCP tool. Always mcp.
        
        :mcp
        
        allowed_tools: Array[String] | McpToolFilter{ read_only, tool_names}
        
        List of allowed tool names or a filter object.
        
        McpAllowedTools = Array[String]
        
        A string array of allowed tool names
        
        class McpToolFilter
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        authorization: String
        
        An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.
        
        connector_id: :connector_dropbox | :connector_gmail | :connector_googlecalendar | 5 more
        
        Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here.
        
        Currently supported connector_id values are:
        
        Dropbox: connector_dropbox
        
        Gmail: connector_gmail
        
        Google Calendar: connector_googlecalendar
        
        Google Drive: connector_googledrive
        
        Microsoft Teams: connector_microsoftteams
        
        Outlook Calendar: connector_outlookcalendar
        
        Outlook Email: connector_outlookemail
        
        SharePoint: connector_sharepoint
        
        :connector_dropbox
        
        :connector_gmail
        
        :connector_googlecalendar
        
        :connector_googledrive
        
        :connector_microsoftteams
        
        :connector_outlookcalendar
        
        :connector_outlookemail
        
        :connector_sharepoint
        
        defer_loading: bool
        
        Whether this MCP tool is deferred and discovered via tool search.
        
        headers: Hash[Symbol, String]
        
        Optional HTTP headers to send to the MCP server. Use for authentication or other purposes.
        
        require_approval: McpToolApprovalFilter{ always, never} | :always | :never
        
        Specify which of the MCP server's tools require approval.
        
        class McpToolApprovalFilter
        
        Specify which of the MCP server's tools require approval. Can be always, never, or a filter object associated with tools that require approval.
        
        always: Always{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        never: Never{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        McpToolApprovalSetting = :always | :never
        
        Specify a single approval policy for all tools. One of always or never. When set to always, all tools will require approval. When set to never, all tools will not require approval.
        
        :always
        
        :never
        
        server_description: String
        
        Optional description of the MCP server, used to provide more context.
        
        server_url: String
        
        The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided.
        
        tunnel_id: String
        
        The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided.
    - tracing: RealtimeTracingConfig
      
      Realtime API can write session traces to the Traces Dashboard. Set to null to disable tracing. Once tracing is enabled for a session, the configuration cannot be modified.
      
      auto will create a trace for the session with default values for the workflow name, group id, and metadata.
      - RealtimeTracingConfig = :auto
        
        Enables tracing and sets default values for tracing configuration options. Always auto.
        
        :auto
      - class TracingConfiguration
        
        Granular configuration for tracing.
        
        group_id: String
        
        The group id to attach to this trace to enable filtering and grouping in the Traces Dashboard.
        
        metadata: untyped
        
        The arbitrary metadata to attach to this trace to enable filtering in the Traces Dashboard.
        
        workflow_name: String
        
        The name of the workflow to attach to this trace. This is used to name the trace in the Traces Dashboard.
    - truncation: RealtimeTruncation
      
      When the number of tokens in a conversation exceeds the model's input token limit, the conversation be truncated, meaning messages (starting from the oldest) will not be included in the model's context. A 32k context model with 4,096 max output tokens can only include 28,224 tokens in the context before truncation occurs.
      
      Clients can configure truncation behavior to truncate with a lower max token limit, which is an effective way to control token usage and cost.
      
      Truncation will reduce the number of cached tokens on the next turn (busting the cache), since messages are dropped from the beginning of the context. However, clients can also configure truncation to retain messages up to a fraction of the maximum context size, which will reduce the need for future truncations and thus improve the cache rate.
      
      Truncation can be disabled entirely, which means the server will never truncate but would instead return an error if the conversation exceeds the model's input token limit.
      - RealtimeTruncationStrategy = :auto | :disabled
        
        The truncation strategy to use for the session. auto is the default truncation strategy. disabled will disable truncation and emit errors when the conversation exceeds the input token limit.
        
        :auto
        
        :disabled
      - class RealtimeTruncationRetentionRatio
        
        Retain a fraction of the conversation tokens when the conversation exceeds the input token limit. This allows you to amortize truncations across multiple turns, which can help improve cached token usage.
        
        retention_ratio: Float
        
        Fraction of post-instruction conversation tokens to retain (0.0 - 1.0) when the conversation exceeds the input token limit. Setting this to 0.8 means that messages will be dropped until 80% of the maximum allowed tokens are used. This helps reduce the frequency of truncations and improve cache rates.
        
        type: :retention_ratio
        
        Use retention ratio truncation.
        
        :retention_ratio
        
        token_limits: TokenLimits{ post_instructions}
        
        Optional custom token limits for this truncation strategy. If not provided, the model's default token limits will be used.
        
        post_instructions: Integer
        
        Maximum tokens allowed in the conversation after instructions (which including tool definitions). For example, setting this to 5,000 would mean that truncation would occur when the conversation exceeds 5,000 tokens after instructions. This cannot be higher than the model's context window size minus the maximum output tokens.
  - class RealtimeTranscriptionSessionCreateRequest
    
    Realtime transcription session object configuration.
    - type: :transcription
      
      The type of session to create. Always transcription for transcription sessions.
      - :transcription
    - audio: RealtimeTranscriptionSessionAudio
      
      Configuration for input and output audio.
      - input: RealtimeTranscriptionSessionAudioInput
        
        format_: RealtimeAudioFormats
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        noise_reduction: NoiseReduction{ type}
        
        Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        transcription: AudioTranscription
        
        Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
        
        turn_detection: RealtimeTranscriptionSessionAudioInputTurnDetection
        
        Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
        
        Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
        
        Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
        
        For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
        
        class ServerVad
        
        Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
        
        type: :server_vad
        
        Type of turn detection, server_vad to turn on simple Server VAD.
        
        :server_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        idle_timeout_ms: Integer
        
        Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
        
        The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
        
        An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        prefix_padding_ms: Integer
        
        Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
        
        silence_duration_ms: Integer
        
        Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
        
        threshold: Float
        
        Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
        
        class SemanticVad
        
        Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
        
        type: :semantic_vad
        
        Type of turn detection, semantic_vad to turn on Semantic VAD.
        
        :semantic_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs.
        
        eagerness: :low | :medium | :high | :auto
        
        Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
        
        :low
        
        :medium
        
        :high
        
        :auto
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.
    - include: Array[:"item.input_audio_transcription.logprobs"]
      
      Additional fields to include in server outputs.
      
      item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
      - :"item.input_audio_transcription.logprobs"
- type: :"session.update"
  
  The event type, must be session.update.
  - :"session.update"
- event_id: String
  
  Optional client-generated ID used to identify this event. This is an arbitrary string that a client may assign. It will be passed back if there is an error with the event, but the corresponding session.updated event will not include it.

Session Updated Event

class SessionUpdatedEvent

Returned when a session is updated with a session.update event, unless there is an error.
- event_id: String
  
  The unique ID of the server event.
- session: RealtimeSessionCreateRequest | RealtimeTranscriptionSessionCreateRequest
  
  The session configuration.
  - class RealtimeSessionCreateRequest
    
    Realtime session object configuration.
    - type: :realtime
      
      The type of session to create. Always realtime for the Realtime API.
      - :realtime
    - audio: RealtimeAudioConfig
      
      Configuration for input and output audio.
      - input: RealtimeAudioConfigInput
        
        format_: RealtimeAudioFormats
        
        The format of the input audio.
        
        class AudioPCM
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
        
        type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
        
        class AudioPCMU
        
        The G.711 μ-law format.
        
        type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
        
        class AudioPCMA
        
        The G.711 A-law format.
        
        type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
        
        noise_reduction: NoiseReduction{ type}
        
        Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
        
        transcription: AudioTranscription
        
        Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
        
        delay: :minimal | :low | :medium | 2 more
        
        Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
        
        :minimal
        
        :low
        
        :medium
        
        :high
        
        :xhigh
        
        language: String
        
        The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
        
        model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        String = String
        
        Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        :"whisper-1"
        
        :"gpt-4o-mini-transcribe"
        
        :"gpt-4o-mini-transcribe-2025-12-15"
        
        :"gpt-4o-transcribe"
        
        :"gpt-4o-transcribe-diarize"
        
        :"gpt-realtime-whisper"
        
        prompt: String
        
        An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
        
        turn_detection: RealtimeAudioInputTurnDetection
        
        Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
        
        Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
        
        Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
        
        For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
        
        class ServerVad
        
        Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
        
        type: :server_vad
        
        Type of turn detection, server_vad to turn on simple Server VAD.
        
        :server_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        idle_timeout_ms: Integer
        
        Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
        
        The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
        
        An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        prefix_padding_ms: Integer
        
        Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
        
        silence_duration_ms: Integer
        
        Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
        
        threshold: Float
        
        Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
        
        class SemanticVad
        
        Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
        
        type: :semantic_vad
        
        Type of turn detection, semantic_vad to turn on Semantic VAD.
        
        :semantic_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs.
        
        eagerness: :low | :medium | :high | :auto
        
        Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
        
        :low
        
        :medium
        
        :high
        
        :auto
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.
      - output: RealtimeAudioConfigOutput
        
        format_: RealtimeAudioFormats
        
        The format of the output audio.
        
        speed: Float
        
        The speed of the model's spoken response as a multiple of the original speed. 1.0 is the default speed. 0.25 is the minimum speed. 1.5 is the maximum speed. This value can only be changed in between model turns, not while a response is in progress.
        
        This parameter is a post-processing adjustment to the audio after it is generated, it's also possible to prompt the model to speak faster or slower.
        
        voice: String | :alloy | :ash | :ballad | 7 more | ID{ id}
        
        The voice the model uses to respond. Supported built-in voices are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. You may also provide a custom voice object with an id, for example { "id": "voice_1234" }. Voice cannot be changed during the session once the model has responded with audio at least once. We recommend marin and cedar for best quality.
        
        String = String
        
        Voice = :alloy | :ash | :ballad | 7 more
        
        :alloy
        
        :ash
        
        :ballad
        
        :coral
        
        :echo
        
        :sage
        
        :shimmer
        
        :verse
        
        :marin
        
        :cedar
        
        class ID
        
        Custom voice reference.
        
        id: String
        
        The custom voice ID, e.g. voice_1234.
    - include: Array[:"item.input_audio_transcription.logprobs"]
      
      Additional fields to include in server outputs.
      
      item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
      - :"item.input_audio_transcription.logprobs"
    - instructions: String
      
      The default system instructions (i.e. system message) prepended to model calls. This field allows the client to guide the model on desired responses. The model can be instructed on response content and format, (e.g. "be extremely succinct", "act friendly", "here are examples of good responses") and on audio behavior (e.g. "talk quickly", "inject emotion into your voice", "laugh frequently"). The instructions are not guaranteed to be followed by the model, but they provide guidance to the model on the desired behavior.
      
      Note that the server sets default instructions which will be used if this field is not set and are visible in the session.created event at the start of the session.
    - max_output_tokens: Integer | :inf
      
      Maximum number of output tokens for a single assistant response, inclusive of tool calls. Provide an integer between 1 and 4096 to limit output tokens, or inf for the maximum available tokens for a given model. Defaults to inf.
      - Integer = Integer
      - MaxOutputTokens = :inf
        
        :inf
    - model: String | :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
      
      The Realtime model used for this session.
      - String = String
      - Model = :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
        
        The Realtime model used for this session.
        
        :"gpt-realtime"
        
        :"gpt-realtime-1.5"
        
        :"gpt-realtime-2"
        
        :"gpt-realtime-2025-08-28"
        
        :"gpt-4o-realtime-preview"
        
        :"gpt-4o-realtime-preview-2024-10-01"
        
        :"gpt-4o-realtime-preview-2024-12-17"
        
        :"gpt-4o-realtime-preview-2025-06-03"
        
        :"gpt-4o-mini-realtime-preview"
        
        :"gpt-4o-mini-realtime-preview-2024-12-17"
        
        :"gpt-realtime-mini"
        
        :"gpt-realtime-mini-2025-10-06"
        
        :"gpt-realtime-mini-2025-12-15"
        
        :"gpt-audio-1.5"
        
        :"gpt-audio-mini"
        
        :"gpt-audio-mini-2025-10-06"
        
        :"gpt-audio-mini-2025-12-15"
    - output_modalities: Array[:text | :audio]
      
      The set of modalities the model can respond with. It defaults to ["audio"], indicating that the model will respond with audio plus a transcript. ["text"] can be used to make the model respond with text only. It is not possible to request both text and audio at the same time.
      - :text
      - :audio
    - parallel_tool_calls: bool
      
      Whether the model may call multiple tools in parallel. Only supported by reasoning Realtime models such as gpt-realtime-2.
    - prompt: ResponsePrompt
      
      Reference to a prompt template and its variables. Learn more.
      - id: String
        
        The unique identifier of the prompt template to use.
      - variables: Hash[Symbol, String | ResponseInputText | ResponseInputImage | ResponseInputFile]
        
        Optional map of values to substitute in for variables in your prompt. The substitution values can either be strings, or other Response input types like images or files.
        
        String = String
        
        class ResponseInputText
        
        A text input to the model.
        
        text: String
        
        The text input to the model.
        
        type: :input_text
        
        The type of the input item. Always input_text.
        
        :input_text
        
        class ResponseInputImage
        
        An image input to the model. Learn about image inputs.
        
        detail: :low | :high | :auto | :original
        
        The detail level of the image to be sent to the model. One of high, low, auto, or original. Defaults to auto.
        
        :low
        
        :high
        
        :auto
        
        :original
        
        type: :input_image
        
        The type of the input item. Always input_image.
        
        :input_image
        
        file_id: String
        
        The ID of the file to be sent to the model.
        
        image_url: String
        
        The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.
        
        class ResponseInputFile
        
        A file input to the model.
        
        type: :input_file
        
        The type of the input item. Always input_file.
        
        :input_file
        
        detail: :low | :high
        
        The detail level of the file to be sent to the model. Use low for the default rendering behavior, or high to render the file at higher quality. Defaults to low.
        
        :low
        
        :high
        
        file_data: String
        
        The content of the file to be sent to the model.
        
        file_id: String
        
        The ID of the file to be sent to the model.
        
        file_url: String
        
        The URL of the file to be sent to the model.
        
        filename: String
        
        The name of the file to be sent to the model.
      - version: String
        
        Optional version of the prompt template.
    - reasoning: RealtimeReasoning
      
      Configuration for reasoning-capable Realtime models such as gpt-realtime-2.
      - effort: RealtimeReasoningEffort
        
        Constrains effort on reasoning for reasoning-capable Realtime models such as gpt-realtime-2.
        
        :minimal
        
        :low
        
        :medium
        
        :high
        
        :xhigh
    - tool_choice: RealtimeToolChoiceConfig
      
      How the model chooses tools. Provide one of the string modes or force a specific function/MCP tool.
      - ToolChoiceOptions = :none | :auto | :required
        
        Controls which (if any) tool is called by the model.
        
        none means the model will not call any tool and instead generates a message.
        
        auto means the model can pick between generating a message or calling one or more tools.
        
        required means the model must call one or more tools.
        
        :none
        
        :auto
        
        :required
      - class ToolChoiceFunction
        
        Use this option to force the model to call a specific function.
        
        name: String
        
        The name of the function to call.
        
        type: :function
        
        For function calling, the type is always function.
        
        :function
      - class ToolChoiceMcp
        
        Use this option to force the model to call a specific tool on a remote MCP server.
        
        server_label: String
        
        The label of the MCP server to use.
        
        type: :mcp
        
        For MCP tools, the type is always mcp.
        
        :mcp
        
        name: String
        
        The name of the tool to call on the server.
    - tools: RealtimeToolsConfig
      
      Tools available to the model.
      - class RealtimeFunctionTool
        
        description: String
        
        The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything).
        
        name: String
        
        The name of the function.
        
        parameters: untyped
        
        Parameters of the function in JSON Schema.
        
        type: :function
        
        The type of the tool, i.e. function.
        
        :function
      - class Mcp
        
        Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.
        
        server_label: String
        
        A label for this MCP server, used to identify it in tool calls.
        
        type: :mcp
        
        The type of the MCP tool. Always mcp.
        
        :mcp
        
        allowed_tools: Array[String] | McpToolFilter{ read_only, tool_names}
        
        List of allowed tool names or a filter object.
        
        McpAllowedTools = Array[String]
        
        A string array of allowed tool names
        
        class McpToolFilter
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        authorization: String
        
        An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.
        
        connector_id: :connector_dropbox | :connector_gmail | :connector_googlecalendar | 5 more
        
        Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here.
        
        Currently supported connector_id values are:
        
        Dropbox: connector_dropbox
        
        Gmail: connector_gmail
        
        Google Calendar: connector_googlecalendar
        
        Google Drive: connector_googledrive
        
        Microsoft Teams: connector_microsoftteams
        
        Outlook Calendar: connector_outlookcalendar
        
        Outlook Email: connector_outlookemail
        
        SharePoint: connector_sharepoint
        
        :connector_dropbox
        
        :connector_gmail
        
        :connector_googlecalendar
        
        :connector_googledrive
        
        :connector_microsoftteams
        
        :connector_outlookcalendar
        
        :connector_outlookemail
        
        :connector_sharepoint
        
        defer_loading: bool
        
        Whether this MCP tool is deferred and discovered via tool search.
        
        headers: Hash[Symbol, String]
        
        Optional HTTP headers to send to the MCP server. Use for authentication or other purposes.
        
        require_approval: McpToolApprovalFilter{ always, never} | :always | :never
        
        Specify which of the MCP server's tools require approval.
        
        class McpToolApprovalFilter
        
        Specify which of the MCP server's tools require approval. Can be always, never, or a filter object associated with tools that require approval.
        
        always: Always{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        never: Never{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        McpToolApprovalSetting = :always | :never
        
        Specify a single approval policy for all tools. One of always or never. When set to always, all tools will require approval. When set to never, all tools will not require approval.
        
        :always
        
        :never
        
        server_description: String
        
        Optional description of the MCP server, used to provide more context.
        
        server_url: String
        
        The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided.
        
        tunnel_id: String
        
        The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided.
    - tracing: RealtimeTracingConfig
      
      Realtime API can write session traces to the Traces Dashboard. Set to null to disable tracing. Once tracing is enabled for a session, the configuration cannot be modified.
      
      auto will create a trace for the session with default values for the workflow name, group id, and metadata.
      - RealtimeTracingConfig = :auto
        
        Enables tracing and sets default values for tracing configuration options. Always auto.
        
        :auto
      - class TracingConfiguration
        
        Granular configuration for tracing.
        
        group_id: String
        
        The group id to attach to this trace to enable filtering and grouping in the Traces Dashboard.
        
        metadata: untyped
        
        The arbitrary metadata to attach to this trace to enable filtering in the Traces Dashboard.
        
        workflow_name: String
        
        The name of the workflow to attach to this trace. This is used to name the trace in the Traces Dashboard.
    - truncation: RealtimeTruncation
      
      When the number of tokens in a conversation exceeds the model's input token limit, the conversation be truncated, meaning messages (starting from the oldest) will not be included in the model's context. A 32k context model with 4,096 max output tokens can only include 28,224 tokens in the context before truncation occurs.
      
      Clients can configure truncation behavior to truncate with a lower max token limit, which is an effective way to control token usage and cost.
      
      Truncation will reduce the number of cached tokens on the next turn (busting the cache), since messages are dropped from the beginning of the context. However, clients can also configure truncation to retain messages up to a fraction of the maximum context size, which will reduce the need for future truncations and thus improve the cache rate.
      
      Truncation can be disabled entirely, which means the server will never truncate but would instead return an error if the conversation exceeds the model's input token limit.
      - RealtimeTruncationStrategy = :auto | :disabled
        
        The truncation strategy to use for the session. auto is the default truncation strategy. disabled will disable truncation and emit errors when the conversation exceeds the input token limit.
        
        :auto
        
        :disabled
      - class RealtimeTruncationRetentionRatio
        
        Retain a fraction of the conversation tokens when the conversation exceeds the input token limit. This allows you to amortize truncations across multiple turns, which can help improve cached token usage.
        
        retention_ratio: Float
        
        Fraction of post-instruction conversation tokens to retain (0.0 - 1.0) when the conversation exceeds the input token limit. Setting this to 0.8 means that messages will be dropped until 80% of the maximum allowed tokens are used. This helps reduce the frequency of truncations and improve cache rates.
        
        type: :retention_ratio
        
        Use retention ratio truncation.
        
        :retention_ratio
        
        token_limits: TokenLimits{ post_instructions}
        
        Optional custom token limits for this truncation strategy. If not provided, the model's default token limits will be used.
        
        post_instructions: Integer
        
        Maximum tokens allowed in the conversation after instructions (which including tool definitions). For example, setting this to 5,000 would mean that truncation would occur when the conversation exceeds 5,000 tokens after instructions. This cannot be higher than the model's context window size minus the maximum output tokens.
  - class RealtimeTranscriptionSessionCreateRequest
    
    Realtime transcription session object configuration.
    - type: :transcription
      
      The type of session to create. Always transcription for transcription sessions.
      - :transcription
    - audio: RealtimeTranscriptionSessionAudio
      
      Configuration for input and output audio.
      - input: RealtimeTranscriptionSessionAudioInput
        
        format_: RealtimeAudioFormats
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        noise_reduction: NoiseReduction{ type}
        
        Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        transcription: AudioTranscription
        
        Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
        
        turn_detection: RealtimeTranscriptionSessionAudioInputTurnDetection
        
        Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
        
        Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
        
        Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
        
        For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
        
        class ServerVad
        
        Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
        
        type: :server_vad
        
        Type of turn detection, server_vad to turn on simple Server VAD.
        
        :server_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        idle_timeout_ms: Integer
        
        Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
        
        The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
        
        An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        prefix_padding_ms: Integer
        
        Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
        
        silence_duration_ms: Integer
        
        Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
        
        threshold: Float
        
        Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
        
        class SemanticVad
        
        Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
        
        type: :semantic_vad
        
        Type of turn detection, semantic_vad to turn on Semantic VAD.
        
        :semantic_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs.
        
        eagerness: :low | :medium | :high | :auto
        
        Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
        
        :low
        
        :medium
        
        :high
        
        :auto
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.
    - include: Array[:"item.input_audio_transcription.logprobs"]
      
      Additional fields to include in server outputs.
      
      item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
      - :"item.input_audio_transcription.logprobs"
- type: :"session.updated"
  
  The event type, must be session.updated.
  - :"session.updated"

Transcription Session Update

class TranscriptionSessionUpdate

Send this event to update a transcription session.
- session: Session{ include, input_audio_format, input_audio_noise_reduction, 2 more}
  
  Realtime transcription session object configuration.
  - include: Array[:"item.input_audio_transcription.logprobs"]
    
    The set of items to include in the transcription. Current available items are: item.input_audio_transcription.logprobs
    - :"item.input_audio_transcription.logprobs"
  - input_audio_format: :pcm16 | :g711_ulaw | :g711_alaw
    
    The format of input audio. Options are pcm16, g711_ulaw, or g711_alaw. For pcm16, input audio must be 16-bit PCM at a 24kHz sample rate, single channel (mono), and little-endian byte order.
    - :pcm16
    - :g711_ulaw
    - :g711_alaw
  - input_audio_noise_reduction: InputAudioNoiseReduction{ type}
    
    Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
    - type: NoiseReductionType
      
      Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
      - :near_field
      - :far_field
  - input_audio_transcription: AudioTranscription
    
    Configuration for input audio transcription. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
    - delay: :minimal | :low | :medium | 2 more
      
      Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
      - :minimal
      - :low
      - :medium
      - :high
      - :xhigh
    - language: String
      
      The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
    - model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
      
      The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
      - String = String
      - Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        :"whisper-1"
        
        :"gpt-4o-mini-transcribe"
        
        :"gpt-4o-mini-transcribe-2025-12-15"
        
        :"gpt-4o-transcribe"
        
        :"gpt-4o-transcribe-diarize"
        
        :"gpt-realtime-whisper"
    - prompt: String
      
      An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
  - turn_detection: TurnDetection{ prefix_padding_ms, silence_duration_ms, threshold, type}
    
    Configuration for turn detection. Can be set to null to turn off. Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
    - prefix_padding_ms: Integer
      
      Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
    - silence_duration_ms: Integer
      
      Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
    - threshold: Float
      
      Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
    - type: :server_vad
      
      Type of turn detection. Only server_vad is currently supported for transcription sessions.
      - :server_vad
- type: :"transcription_session.update"
  
  The event type, must be transcription_session.update.
  - :"transcription_session.update"
- event_id: String
  
  Optional client-generated ID used to identify this event.

Transcription Session Updated Event

class TranscriptionSessionUpdatedEvent

Returned when a transcription session is updated with a transcription_session.update event, unless there is an error.
- event_id: String
  
  The unique ID of the server event.
- session: Session{ client_secret, input_audio_format, input_audio_transcription, 2 more}
  
  A new Realtime transcription session configuration.
  
  When a session is created on the server via REST API, the session object also contains an ephemeral key. Default TTL for keys is 10 minutes. This property is not present when a session is updated via the WebSocket API.
  - client_secret: ClientSecret{ expires_at, value}
    
    Ephemeral key returned by the API. Only present when the session is created on the server via REST API.
    - expires_at: Integer
      
      Timestamp for when the token expires. Currently, all tokens expire after one minute.
    - value: String
      
      Ephemeral key usable in client environments to authenticate connections to the Realtime API. Use this in client-side environments rather than a standard API token, which should only be used server-side.
  - input_audio_format: String
    
    The format of input audio. Options are pcm16, g711_ulaw, or g711_alaw.
  - input_audio_transcription: AudioTranscription
    - delay: :minimal | :low | :medium | 2 more
      
      Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
      - :minimal
      - :low
      - :medium
      - :high
      - :xhigh
    - language: String
      
      The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
    - model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
      
      The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
      - String = String
      - Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        :"whisper-1"
        
        :"gpt-4o-mini-transcribe"
        
        :"gpt-4o-mini-transcribe-2025-12-15"
        
        :"gpt-4o-transcribe"
        
        :"gpt-4o-transcribe-diarize"
        
        :"gpt-realtime-whisper"
    - prompt: String
      
      An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
  - modalities: Array[:text | :audio]
    
    The set of modalities the model can respond with. To disable audio, set this to ["text"].
    - :text
    - :audio
  - turn_detection: TurnDetection{ prefix_padding_ms, silence_duration_ms, threshold, type}
    
    Configuration for turn detection. Can be set to null to turn off. Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
    - prefix_padding_ms: Integer
      
      Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
    - silence_duration_ms: Integer
      
      Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
    - threshold: Float
      
      Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
    - type: String
      
      Type of turn detection, only server_vad is currently supported.
- type: :"transcription_session.updated"
  
  The event type, must be transcription_session.updated.
  - :"transcription_session.updated"

Client Secrets

Create client secret

realtime.client_secrets.create(**kwargs) -> ClientSecretCreateResponse

post /realtime/client_secrets

Create a Realtime client secret with an associated session configuration.

Client secrets are short-lived tokens that can be passed to a client app, such as a web frontend or mobile client, which grants access to the Realtime API without leaking your main API key. You can configure a custom TTL for each client secret.

You can also attach session configuration options to the client secret, which will be applied to any sessions created using that client secret, but these can also be overridden by the client connection.

Learn more about authentication with client secrets over WebRTC.

Returns the created client secret and the effective session object. The client secret is a string that looks like ek_1234.

Parameters

expires_after: ExpiresAfter{ anchor, seconds}

Configuration for the client secret expiration. Expiration refers to the time after which a client secret will no longer be valid for creating sessions. The session itself may continue after that time once started. A secret can be used to create multiple sessions until it expires.
- anchor: :created_at
  
  The anchor point for the client secret expiration, meaning that seconds will be added to the created_at time of the client secret to produce an expiration timestamp. Only created_at is currently supported.
  - :created_at
- seconds: Integer
  
  The number of seconds from the anchor point to the expiration. Select a value between 10 and 7200 (2 hours). This default to 600 seconds (10 minutes) if not specified.
session: RealtimeSessionCreateRequest | RealtimeTranscriptionSessionCreateRequest

Session configuration to use for the client secret. Choose either a realtime session or a transcription session.
- class RealtimeSessionCreateRequest
  
  Realtime session object configuration.
  - type: :realtime
    
    The type of session to create. Always realtime for the Realtime API.
    - :realtime
  - audio: RealtimeAudioConfig
    
    Configuration for input and output audio.
    - input: RealtimeAudioConfigInput
      - format_: RealtimeAudioFormats
        
        The format of the input audio.
        
        class AudioPCM
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
        
        type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
        
        class AudioPCMU
        
        The G.711 μ-law format.
        
        type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
        
        class AudioPCMA
        
        The G.711 A-law format.
        
        type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
      - noise_reduction: NoiseReduction{ type}
        
        Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
      - transcription: AudioTranscription
        
        Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
        
        delay: :minimal | :low | :medium | 2 more
        
        Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
        
        :minimal
        
        :low
        
        :medium
        
        :high
        
        :xhigh
        
        language: String
        
        The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
        
        model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        String = String
        
        Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        :"whisper-1"
        
        :"gpt-4o-mini-transcribe"
        
        :"gpt-4o-mini-transcribe-2025-12-15"
        
        :"gpt-4o-transcribe"
        
        :"gpt-4o-transcribe-diarize"
        
        :"gpt-realtime-whisper"
        
        prompt: String
        
        An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
      - turn_detection: RealtimeAudioInputTurnDetection
        
        Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
        
        Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
        
        Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
        
        For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
        
        class ServerVad
        
        Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
        
        type: :server_vad
        
        Type of turn detection, server_vad to turn on simple Server VAD.
        
        :server_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        idle_timeout_ms: Integer
        
        Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
        
        The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
        
        An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        prefix_padding_ms: Integer
        
        Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
        
        silence_duration_ms: Integer
        
        Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
        
        threshold: Float
        
        Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
        
        class SemanticVad
        
        Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
        
        type: :semantic_vad
        
        Type of turn detection, semantic_vad to turn on Semantic VAD.
        
        :semantic_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs.
        
        eagerness: :low | :medium | :high | :auto
        
        Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
        
        :low
        
        :medium
        
        :high
        
        :auto
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.
    - output: RealtimeAudioConfigOutput
      - format_: RealtimeAudioFormats
        
        The format of the output audio.
      - speed: Float
        
        The speed of the model's spoken response as a multiple of the original speed. 1.0 is the default speed. 0.25 is the minimum speed. 1.5 is the maximum speed. This value can only be changed in between model turns, not while a response is in progress.
        
        This parameter is a post-processing adjustment to the audio after it is generated, it's also possible to prompt the model to speak faster or slower.
      - voice: String | :alloy | :ash | :ballad | 7 more | ID{ id}
        
        The voice the model uses to respond. Supported built-in voices are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. You may also provide a custom voice object with an id, for example { "id": "voice_1234" }. Voice cannot be changed during the session once the model has responded with audio at least once. We recommend marin and cedar for best quality.
        
        String = String
        
        Voice = :alloy | :ash | :ballad | 7 more
        
        :alloy
        
        :ash
        
        :ballad
        
        :coral
        
        :echo
        
        :sage
        
        :shimmer
        
        :verse
        
        :marin
        
        :cedar
        
        class ID
        
        Custom voice reference.
        
        id: String
        
        The custom voice ID, e.g. voice_1234.
  - include: Array[:"item.input_audio_transcription.logprobs"]
    
    Additional fields to include in server outputs.
    
    item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
    - :"item.input_audio_transcription.logprobs"
  - instructions: String
    
    The default system instructions (i.e. system message) prepended to model calls. This field allows the client to guide the model on desired responses. The model can be instructed on response content and format, (e.g. "be extremely succinct", "act friendly", "here are examples of good responses") and on audio behavior (e.g. "talk quickly", "inject emotion into your voice", "laugh frequently"). The instructions are not guaranteed to be followed by the model, but they provide guidance to the model on the desired behavior.
    
    Note that the server sets default instructions which will be used if this field is not set and are visible in the session.created event at the start of the session.
  - max_output_tokens: Integer | :inf
    
    Maximum number of output tokens for a single assistant response, inclusive of tool calls. Provide an integer between 1 and 4096 to limit output tokens, or inf for the maximum available tokens for a given model. Defaults to inf.
    - Integer = Integer
    - MaxOutputTokens = :inf
      - :inf
  - model: String | :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
    
    The Realtime model used for this session.
    - String = String
    - Model = :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
      
      The Realtime model used for this session.
      - :"gpt-realtime"
      - :"gpt-realtime-1.5"
      - :"gpt-realtime-2"
      - :"gpt-realtime-2025-08-28"
      - :"gpt-4o-realtime-preview"
      - :"gpt-4o-realtime-preview-2024-10-01"
      - :"gpt-4o-realtime-preview-2024-12-17"
      - :"gpt-4o-realtime-preview-2025-06-03"
      - :"gpt-4o-mini-realtime-preview"
      - :"gpt-4o-mini-realtime-preview-2024-12-17"
      - :"gpt-realtime-mini"
      - :"gpt-realtime-mini-2025-10-06"
      - :"gpt-realtime-mini-2025-12-15"
      - :"gpt-audio-1.5"
      - :"gpt-audio-mini"
      - :"gpt-audio-mini-2025-10-06"
      - :"gpt-audio-mini-2025-12-15"
  - output_modalities: Array[:text | :audio]
    
    The set of modalities the model can respond with. It defaults to ["audio"], indicating that the model will respond with audio plus a transcript. ["text"] can be used to make the model respond with text only. It is not possible to request both text and audio at the same time.
    - :text
    - :audio
  - parallel_tool_calls: bool
    
    Whether the model may call multiple tools in parallel. Only supported by reasoning Realtime models such as gpt-realtime-2.
  - prompt: ResponsePrompt
    
    Reference to a prompt template and its variables. Learn more.
    - id: String
      
      The unique identifier of the prompt template to use.
    - variables: Hash[Symbol, String | ResponseInputText | ResponseInputImage | ResponseInputFile]
      
      Optional map of values to substitute in for variables in your prompt. The substitution values can either be strings, or other Response input types like images or files.
      - String = String
      - class ResponseInputText
        
        A text input to the model.
        
        text: String
        
        The text input to the model.
        
        type: :input_text
        
        The type of the input item. Always input_text.
        
        :input_text
      - class ResponseInputImage
        
        An image input to the model. Learn about image inputs.
        
        detail: :low | :high | :auto | :original
        
        The detail level of the image to be sent to the model. One of high, low, auto, or original. Defaults to auto.
        
        :low
        
        :high
        
        :auto
        
        :original
        
        type: :input_image
        
        The type of the input item. Always input_image.
        
        :input_image
        
        file_id: String
        
        The ID of the file to be sent to the model.
        
        image_url: String
        
        The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.
      - class ResponseInputFile
        
        A file input to the model.
        
        type: :input_file
        
        The type of the input item. Always input_file.
        
        :input_file
        
        detail: :low | :high
        
        The detail level of the file to be sent to the model. Use low for the default rendering behavior, or high to render the file at higher quality. Defaults to low.
        
        :low
        
        :high
        
        file_data: String
        
        The content of the file to be sent to the model.
        
        file_id: String
        
        The ID of the file to be sent to the model.
        
        file_url: String
        
        The URL of the file to be sent to the model.
        
        filename: String
        
        The name of the file to be sent to the model.
    - version: String
      
      Optional version of the prompt template.
  - reasoning: RealtimeReasoning
    
    Configuration for reasoning-capable Realtime models such as gpt-realtime-2.
    - effort: RealtimeReasoningEffort
      
      Constrains effort on reasoning for reasoning-capable Realtime models such as gpt-realtime-2.
      - :minimal
      - :low
      - :medium
      - :high
      - :xhigh
  - tool_choice: RealtimeToolChoiceConfig
    
    How the model chooses tools. Provide one of the string modes or force a specific function/MCP tool.
    - ToolChoiceOptions = :none | :auto | :required
      
      Controls which (if any) tool is called by the model.
      
      none means the model will not call any tool and instead generates a message.
      
      auto means the model can pick between generating a message or calling one or more tools.
      
      required means the model must call one or more tools.
      - :none
      - :auto
      - :required
    - class ToolChoiceFunction
      
      Use this option to force the model to call a specific function.
      - name: String
        
        The name of the function to call.
      - type: :function
        
        For function calling, the type is always function.
        
        :function
    - class ToolChoiceMcp
      
      Use this option to force the model to call a specific tool on a remote MCP server.
      - server_label: String
        
        The label of the MCP server to use.
      - type: :mcp
        
        For MCP tools, the type is always mcp.
        
        :mcp
      - name: String
        
        The name of the tool to call on the server.
  - tools: RealtimeToolsConfig
    
    Tools available to the model.
    - class RealtimeFunctionTool
      - description: String
        
        The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything).
      - name: String
        
        The name of the function.
      - parameters: untyped
        
        Parameters of the function in JSON Schema.
      - type: :function
        
        The type of the tool, i.e. function.
        
        :function
    - class Mcp
      
      Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.
      - server_label: String
        
        A label for this MCP server, used to identify it in tool calls.
      - type: :mcp
        
        The type of the MCP tool. Always mcp.
        
        :mcp
      - allowed_tools: Array[String] | McpToolFilter{ read_only, tool_names}
        
        List of allowed tool names or a filter object.
        
        McpAllowedTools = Array[String]
        
        A string array of allowed tool names
        
        class McpToolFilter
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
      - authorization: String
        
        An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.
      - connector_id: :connector_dropbox | :connector_gmail | :connector_googlecalendar | 5 more
        
        Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here.
        
        Currently supported connector_id values are:
        
        Dropbox: connector_dropbox
        
        Gmail: connector_gmail
        
        Google Calendar: connector_googlecalendar
        
        Google Drive: connector_googledrive
        
        Microsoft Teams: connector_microsoftteams
        
        Outlook Calendar: connector_outlookcalendar
        
        Outlook Email: connector_outlookemail
        
        SharePoint: connector_sharepoint
        
        :connector_dropbox
        
        :connector_gmail
        
        :connector_googlecalendar
        
        :connector_googledrive
        
        :connector_microsoftteams
        
        :connector_outlookcalendar
        
        :connector_outlookemail
        
        :connector_sharepoint
      - defer_loading: bool
        
        Whether this MCP tool is deferred and discovered via tool search.
      - headers: Hash[Symbol, String]
        
        Optional HTTP headers to send to the MCP server. Use for authentication or other purposes.
      - require_approval: McpToolApprovalFilter{ always, never} | :always | :never
        
        Specify which of the MCP server's tools require approval.
        
        class McpToolApprovalFilter
        
        Specify which of the MCP server's tools require approval. Can be always, never, or a filter object associated with tools that require approval.
        
        always: Always{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        never: Never{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        McpToolApprovalSetting = :always | :never
        
        Specify a single approval policy for all tools. One of always or never. When set to always, all tools will require approval. When set to never, all tools will not require approval.
        
        :always
        
        :never
      - server_description: String
        
        Optional description of the MCP server, used to provide more context.
      - server_url: String
        
        The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided.
      - tunnel_id: String
        
        The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided.
  - tracing: RealtimeTracingConfig
    
    Realtime API can write session traces to the Traces Dashboard. Set to null to disable tracing. Once tracing is enabled for a session, the configuration cannot be modified.
    
    auto will create a trace for the session with default values for the workflow name, group id, and metadata.
    - RealtimeTracingConfig = :auto
      
      Enables tracing and sets default values for tracing configuration options. Always auto.
      - :auto
    - class TracingConfiguration
      
      Granular configuration for tracing.
      - group_id: String
        
        The group id to attach to this trace to enable filtering and grouping in the Traces Dashboard.
      - metadata: untyped
        
        The arbitrary metadata to attach to this trace to enable filtering in the Traces Dashboard.
      - workflow_name: String
        
        The name of the workflow to attach to this trace. This is used to name the trace in the Traces Dashboard.
  - truncation: RealtimeTruncation
    
    When the number of tokens in a conversation exceeds the model's input token limit, the conversation be truncated, meaning messages (starting from the oldest) will not be included in the model's context. A 32k context model with 4,096 max output tokens can only include 28,224 tokens in the context before truncation occurs.
    
    Clients can configure truncation behavior to truncate with a lower max token limit, which is an effective way to control token usage and cost.
    
    Truncation will reduce the number of cached tokens on the next turn (busting the cache), since messages are dropped from the beginning of the context. However, clients can also configure truncation to retain messages up to a fraction of the maximum context size, which will reduce the need for future truncations and thus improve the cache rate.
    
    Truncation can be disabled entirely, which means the server will never truncate but would instead return an error if the conversation exceeds the model's input token limit.
    - RealtimeTruncationStrategy = :auto | :disabled
      
      The truncation strategy to use for the session. auto is the default truncation strategy. disabled will disable truncation and emit errors when the conversation exceeds the input token limit.
      - :auto
      - :disabled
    - class RealtimeTruncationRetentionRatio
      
      Retain a fraction of the conversation tokens when the conversation exceeds the input token limit. This allows you to amortize truncations across multiple turns, which can help improve cached token usage.
      - retention_ratio: Float
        
        Fraction of post-instruction conversation tokens to retain (0.0 - 1.0) when the conversation exceeds the input token limit. Setting this to 0.8 means that messages will be dropped until 80% of the maximum allowed tokens are used. This helps reduce the frequency of truncations and improve cache rates.
      - type: :retention_ratio
        
        Use retention ratio truncation.
        
        :retention_ratio
      - token_limits: TokenLimits{ post_instructions}
        
        Optional custom token limits for this truncation strategy. If not provided, the model's default token limits will be used.
        
        post_instructions: Integer
        
        Maximum tokens allowed in the conversation after instructions (which including tool definitions). For example, setting this to 5,000 would mean that truncation would occur when the conversation exceeds 5,000 tokens after instructions. This cannot be higher than the model's context window size minus the maximum output tokens.
- class RealtimeTranscriptionSessionCreateRequest
  
  Realtime transcription session object configuration.
  - type: :transcription
    
    The type of session to create. Always transcription for transcription sessions.
    - :transcription
  - audio: RealtimeTranscriptionSessionAudio
    
    Configuration for input and output audio.
    - input: RealtimeTranscriptionSessionAudioInput
      - format_: RealtimeAudioFormats
        
        The PCM audio format. Only a 24kHz sample rate is supported.
      - noise_reduction: NoiseReduction{ type}
        
        Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
      - transcription: AudioTranscription
        
        Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
      - turn_detection: RealtimeTranscriptionSessionAudioInputTurnDetection
        
        Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
        
        Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
        
        Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
        
        For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
        
        class ServerVad
        
        Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
        
        type: :server_vad
        
        Type of turn detection, server_vad to turn on simple Server VAD.
        
        :server_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        idle_timeout_ms: Integer
        
        Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
        
        The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
        
        An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        prefix_padding_ms: Integer
        
        Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
        
        silence_duration_ms: Integer
        
        Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
        
        threshold: Float
        
        Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
        
        class SemanticVad
        
        Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
        
        type: :semantic_vad
        
        Type of turn detection, semantic_vad to turn on Semantic VAD.
        
        :semantic_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs.
        
        eagerness: :low | :medium | :high | :auto
        
        Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
        
        :low
        
        :medium
        
        :high
        
        :auto
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.
  - include: Array[:"item.input_audio_transcription.logprobs"]
    
    Additional fields to include in server outputs.
    
    item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
    - :"item.input_audio_transcription.logprobs"

Returns

class ClientSecretCreateResponse

Response from creating a session and client secret for the Realtime API.
- expires_at: Integer
  
  Expiration timestamp for the client secret, in seconds since epoch.
- session: RealtimeSessionCreateResponse | RealtimeTranscriptionSessionCreateResponse
  
  The session configuration for either a realtime or transcription session.
  - class RealtimeSessionCreateResponse
    
    A Realtime session configuration object.
    - id: String
      
      Unique identifier for the session that looks like sess_1234567890abcdef.
    - object: :"realtime.session"
      
      The object type. Always realtime.session.
      - :"realtime.session"
    - type: :realtime
      
      The type of session to create. Always realtime for the Realtime API.
      - :realtime
    - audio: Audio{ input, output}
      
      Configuration for input and output audio.
      - input: Input{ format_, noise_reduction, transcription, turn_detection}
        
        format_: RealtimeAudioFormats
        
        The format of the input audio.
        
        class AudioPCM
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
        
        type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
        
        class AudioPCMU
        
        The G.711 μ-law format.
        
        type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
        
        class AudioPCMA
        
        The G.711 A-law format.
        
        type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
        
        noise_reduction: NoiseReduction{ type}
        
        Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
        
        transcription: AudioTranscription
        
        delay: :minimal | :low | :medium | 2 more
        
        Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
        
        :minimal
        
        :low
        
        :medium
        
        :high
        
        :xhigh
        
        language: String
        
        The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
        
        model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        String = String
        
        Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        :"whisper-1"
        
        :"gpt-4o-mini-transcribe"
        
        :"gpt-4o-mini-transcribe-2025-12-15"
        
        :"gpt-4o-transcribe"
        
        :"gpt-4o-transcribe-diarize"
        
        :"gpt-realtime-whisper"
        
        prompt: String
        
        An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
        
        turn_detection: ServerVad{ type, create_response, idle_timeout_ms, 4 more} | SemanticVad{ type, create_response, eagerness, interrupt_response}
        
        Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
        
        Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
        
        Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
        
        For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
        
        class ServerVad
        
        Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
        
        type: :server_vad
        
        Type of turn detection, server_vad to turn on simple Server VAD.
        
        :server_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        idle_timeout_ms: Integer
        
        Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
        
        The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
        
        An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        prefix_padding_ms: Integer
        
        Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
        
        silence_duration_ms: Integer
        
        Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
        
        threshold: Float
        
        Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
        
        class SemanticVad
        
        Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
        
        type: :semantic_vad
        
        Type of turn detection, semantic_vad to turn on Semantic VAD.
        
        :semantic_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs.
        
        eagerness: :low | :medium | :high | :auto
        
        Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
        
        :low
        
        :medium
        
        :high
        
        :auto
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.
      - output: Output{ format_, speed, voice}
        
        format_: RealtimeAudioFormats
        
        The format of the output audio.
        
        speed: Float
        
        The speed of the model's spoken response as a multiple of the original speed. 1.0 is the default speed. 0.25 is the minimum speed. 1.5 is the maximum speed. This value can only be changed in between model turns, not while a response is in progress.
        
        This parameter is a post-processing adjustment to the audio after it is generated, it's also possible to prompt the model to speak faster or slower.
        
        voice: String | :alloy | :ash | :ballad | 7 more
        
        The voice the model uses to respond. Voice cannot be changed during the session once the model has responded with audio at least once. Current voice options are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. We recommend marin and cedar for best quality.
        
        String = String
        
        Voice = :alloy | :ash | :ballad | 7 more
        
        The voice the model uses to respond. Voice cannot be changed during the session once the model has responded with audio at least once. Current voice options are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. We recommend marin and cedar for best quality.
        
        :alloy
        
        :ash
        
        :ballad
        
        :coral
        
        :echo
        
        :sage
        
        :shimmer
        
        :verse
        
        :marin
        
        :cedar
    - expires_at: Integer
      
      Expiration timestamp for the session, in seconds since epoch.
    - include: Array[:"item.input_audio_transcription.logprobs"]
      
      Additional fields to include in server outputs.
      
      item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
      - :"item.input_audio_transcription.logprobs"
    - instructions: String
      
      The default system instructions (i.e. system message) prepended to model calls. This field allows the client to guide the model on desired responses. The model can be instructed on response content and format, (e.g. "be extremely succinct", "act friendly", "here are examples of good responses") and on audio behavior (e.g. "talk quickly", "inject emotion into your voice", "laugh frequently"). The instructions are not guaranteed to be followed by the model, but they provide guidance to the model on the desired behavior.
      
      Note that the server sets default instructions which will be used if this field is not set and are visible in the session.created event at the start of the session.
    - max_output_tokens: Integer | :inf
      
      Maximum number of output tokens for a single assistant response, inclusive of tool calls. Provide an integer between 1 and 4096 to limit output tokens, or inf for the maximum available tokens for a given model. Defaults to inf.
      - Integer = Integer
      - MaxOutputTokens = :inf
        
        :inf
    - model: String | :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
      
      The Realtime model used for this session.
      - String = String
      - Model = :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
        
        The Realtime model used for this session.
        
        :"gpt-realtime"
        
        :"gpt-realtime-1.5"
        
        :"gpt-realtime-2"
        
        :"gpt-realtime-2025-08-28"
        
        :"gpt-4o-realtime-preview"
        
        :"gpt-4o-realtime-preview-2024-10-01"
        
        :"gpt-4o-realtime-preview-2024-12-17"
        
        :"gpt-4o-realtime-preview-2025-06-03"
        
        :"gpt-4o-mini-realtime-preview"
        
        :"gpt-4o-mini-realtime-preview-2024-12-17"
        
        :"gpt-realtime-mini"
        
        :"gpt-realtime-mini-2025-10-06"
        
        :"gpt-realtime-mini-2025-12-15"
        
        :"gpt-audio-1.5"
        
        :"gpt-audio-mini"
        
        :"gpt-audio-mini-2025-10-06"
        
        :"gpt-audio-mini-2025-12-15"
    - output_modalities: Array[:text | :audio]
      
      The set of modalities the model can respond with. It defaults to ["audio"], indicating that the model will respond with audio plus a transcript. ["text"] can be used to make the model respond with text only. It is not possible to request both text and audio at the same time.
      - :text
      - :audio
    - prompt: ResponsePrompt
      
      Reference to a prompt template and its variables. Learn more.
      - id: String
        
        The unique identifier of the prompt template to use.
      - variables: Hash[Symbol, String | ResponseInputText | ResponseInputImage | ResponseInputFile]
        
        Optional map of values to substitute in for variables in your prompt. The substitution values can either be strings, or other Response input types like images or files.
        
        String = String
        
        class ResponseInputText
        
        A text input to the model.
        
        text: String
        
        The text input to the model.
        
        type: :input_text
        
        The type of the input item. Always input_text.
        
        :input_text
        
        class ResponseInputImage
        
        An image input to the model. Learn about image inputs.
        
        detail: :low | :high | :auto | :original
        
        The detail level of the image to be sent to the model. One of high, low, auto, or original. Defaults to auto.
        
        :low
        
        :high
        
        :auto
        
        :original
        
        type: :input_image
        
        The type of the input item. Always input_image.
        
        :input_image
        
        file_id: String
        
        The ID of the file to be sent to the model.
        
        image_url: String
        
        The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.
        
        class ResponseInputFile
        
        A file input to the model.
        
        type: :input_file
        
        The type of the input item. Always input_file.
        
        :input_file
        
        detail: :low | :high
        
        The detail level of the file to be sent to the model. Use low for the default rendering behavior, or high to render the file at higher quality. Defaults to low.
        
        :low
        
        :high
        
        file_data: String
        
        The content of the file to be sent to the model.
        
        file_id: String
        
        The ID of the file to be sent to the model.
        
        file_url: String
        
        The URL of the file to be sent to the model.
        
        filename: String
        
        The name of the file to be sent to the model.
      - version: String
        
        Optional version of the prompt template.
    - reasoning: RealtimeReasoning
      
      Configuration for reasoning-capable Realtime models such as gpt-realtime-2.
      - effort: RealtimeReasoningEffort
        
        Constrains effort on reasoning for reasoning-capable Realtime models such as gpt-realtime-2.
        
        :minimal
        
        :low
        
        :medium
        
        :high
        
        :xhigh
    - tool_choice: ToolChoiceOptions | ToolChoiceFunction | ToolChoiceMcp
      
      How the model chooses tools. Provide one of the string modes or force a specific function/MCP tool.
      - ToolChoiceOptions = :none | :auto | :required
        
        Controls which (if any) tool is called by the model.
        
        none means the model will not call any tool and instead generates a message.
        
        auto means the model can pick between generating a message or calling one or more tools.
        
        required means the model must call one or more tools.
        
        :none
        
        :auto
        
        :required
      - class ToolChoiceFunction
        
        Use this option to force the model to call a specific function.
        
        name: String
        
        The name of the function to call.
        
        type: :function
        
        For function calling, the type is always function.
        
        :function
      - class ToolChoiceMcp
        
        Use this option to force the model to call a specific tool on a remote MCP server.
        
        server_label: String
        
        The label of the MCP server to use.
        
        type: :mcp
        
        For MCP tools, the type is always mcp.
        
        :mcp
        
        name: String
        
        The name of the tool to call on the server.
    - tools: Array[RealtimeFunctionTool | McpTool{ server_label, type, allowed_tools, 8 more}]
      
      Tools available to the model.
      - class RealtimeFunctionTool
        
        description: String
        
        The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything).
        
        name: String
        
        The name of the function.
        
        parameters: untyped
        
        Parameters of the function in JSON Schema.
        
        type: :function
        
        The type of the tool, i.e. function.
        
        :function
      - class McpTool
        
        Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.
        
        server_label: String
        
        A label for this MCP server, used to identify it in tool calls.
        
        type: :mcp
        
        The type of the MCP tool. Always mcp.
        
        :mcp
        
        allowed_tools: Array[String] | McpToolFilter{ read_only, tool_names}
        
        List of allowed tool names or a filter object.
        
        McpAllowedTools = Array[String]
        
        A string array of allowed tool names
        
        class McpToolFilter
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        authorization: String
        
        An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.
        
        connector_id: :connector_dropbox | :connector_gmail | :connector_googlecalendar | 5 more
        
        Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here.
        
        Currently supported connector_id values are:
        
        Dropbox: connector_dropbox
        
        Gmail: connector_gmail
        
        Google Calendar: connector_googlecalendar
        
        Google Drive: connector_googledrive
        
        Microsoft Teams: connector_microsoftteams
        
        Outlook Calendar: connector_outlookcalendar
        
        Outlook Email: connector_outlookemail
        
        SharePoint: connector_sharepoint
        
        :connector_dropbox
        
        :connector_gmail
        
        :connector_googlecalendar
        
        :connector_googledrive
        
        :connector_microsoftteams
        
        :connector_outlookcalendar
        
        :connector_outlookemail
        
        :connector_sharepoint
        
        defer_loading: bool
        
        Whether this MCP tool is deferred and discovered via tool search.
        
        headers: Hash[Symbol, String]
        
        Optional HTTP headers to send to the MCP server. Use for authentication or other purposes.
        
        require_approval: McpToolApprovalFilter{ always, never} | :always | :never
        
        Specify which of the MCP server's tools require approval.
        
        class McpToolApprovalFilter
        
        Specify which of the MCP server's tools require approval. Can be always, never, or a filter object associated with tools that require approval.
        
        always: Always{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        never: Never{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        McpToolApprovalSetting = :always | :never
        
        Specify a single approval policy for all tools. One of always or never. When set to always, all tools will require approval. When set to never, all tools will not require approval.
        
        :always
        
        :never
        
        server_description: String
        
        Optional description of the MCP server, used to provide more context.
        
        server_url: String
        
        The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided.
        
        tunnel_id: String
        
        The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided.
    - tracing: :auto | TracingConfiguration{ group_id, metadata, workflow_name}
      
      Realtime API can write session traces to the Traces Dashboard. Set to null to disable tracing. Once tracing is enabled for a session, the configuration cannot be modified.
      
      auto will create a trace for the session with default values for the workflow name, group id, and metadata.
      - Tracing = :auto
        
        Enables tracing and sets default values for tracing configuration options. Always auto.
        
        :auto
      - class TracingConfiguration
        
        Granular configuration for tracing.
        
        group_id: String
        
        The group id to attach to this trace to enable filtering and grouping in the Traces Dashboard.
        
        metadata: untyped
        
        The arbitrary metadata to attach to this trace to enable filtering in the Traces Dashboard.
        
        workflow_name: String
        
        The name of the workflow to attach to this trace. This is used to name the trace in the Traces Dashboard.
    - truncation: RealtimeTruncation
      
      When the number of tokens in a conversation exceeds the model's input token limit, the conversation be truncated, meaning messages (starting from the oldest) will not be included in the model's context. A 32k context model with 4,096 max output tokens can only include 28,224 tokens in the context before truncation occurs.
      
      Clients can configure truncation behavior to truncate with a lower max token limit, which is an effective way to control token usage and cost.
      
      Truncation will reduce the number of cached tokens on the next turn (busting the cache), since messages are dropped from the beginning of the context. However, clients can also configure truncation to retain messages up to a fraction of the maximum context size, which will reduce the need for future truncations and thus improve the cache rate.
      
      Truncation can be disabled entirely, which means the server will never truncate but would instead return an error if the conversation exceeds the model's input token limit.
      - RealtimeTruncationStrategy = :auto | :disabled
        
        The truncation strategy to use for the session. auto is the default truncation strategy. disabled will disable truncation and emit errors when the conversation exceeds the input token limit.
        
        :auto
        
        :disabled
      - class RealtimeTruncationRetentionRatio
        
        Retain a fraction of the conversation tokens when the conversation exceeds the input token limit. This allows you to amortize truncations across multiple turns, which can help improve cached token usage.
        
        retention_ratio: Float
        
        Fraction of post-instruction conversation tokens to retain (0.0 - 1.0) when the conversation exceeds the input token limit. Setting this to 0.8 means that messages will be dropped until 80% of the maximum allowed tokens are used. This helps reduce the frequency of truncations and improve cache rates.
        
        type: :retention_ratio
        
        Use retention ratio truncation.
        
        :retention_ratio
        
        token_limits: TokenLimits{ post_instructions}
        
        Optional custom token limits for this truncation strategy. If not provided, the model's default token limits will be used.
        
        post_instructions: Integer
        
        Maximum tokens allowed in the conversation after instructions (which including tool definitions). For example, setting this to 5,000 would mean that truncation would occur when the conversation exceeds 5,000 tokens after instructions. This cannot be higher than the model's context window size minus the maximum output tokens.
  - class RealtimeTranscriptionSessionCreateResponse
    
    A Realtime transcription session configuration object.
    - id: String
      
      Unique identifier for the session that looks like sess_1234567890abcdef.
    - object: String
      
      The object type. Always realtime.transcription_session.
    - type: :transcription
      
      The type of session. Always transcription for transcription sessions.
      - :transcription
    - audio: Audio{ input}
      
      Configuration for input audio for the session.
      - input: Input{ format_, noise_reduction, transcription, turn_detection}
        
        format_: RealtimeAudioFormats
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        noise_reduction: NoiseReduction{ type}
        
        Configuration for input audio noise reduction.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        transcription: AudioTranscription
        
        turn_detection: RealtimeTranscriptionSessionTurnDetection
        
        Configuration for turn detection. Can be set to null to turn off. Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech. For gpt-realtime-whisper, this must be null; VAD is not supported.
        
        prefix_padding_ms: Integer
        
        Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
        
        silence_duration_ms: Integer
        
        Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
        
        threshold: Float
        
        Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
        
        type: String
        
        Type of turn detection, only server_vad is currently supported.
    - expires_at: Integer
      
      Expiration timestamp for the session, in seconds since epoch.
    - include: Array[:"item.input_audio_transcription.logprobs"]
      
      Additional fields to include in server outputs.
      - item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
      - :"item.input_audio_transcription.logprobs"
- value: String
  
  The generated client secret value.

Example

require "openai"

openai = OpenAI::Client.new(api_key: "My API Key")

client_secret = openai.realtime.client_secrets.create

puts(client_secret)

Response

{
  "expires_at": 0,
  "session": {
    "id": "id",
    "object": "realtime.session",
    "type": "realtime",
    "audio": {
      "input": {
        "format": {
          "rate": 24000,
          "type": "audio/pcm"
        },
        "noise_reduction": {
          "type": "near_field"
        },
        "transcription": {
          "delay": "minimal",
          "language": "language",
          "model": "whisper-1",
          "prompt": "prompt"
        },
        "turn_detection": {
          "type": "server_vad",
          "create_response": true,
          "idle_timeout_ms": 5000,
          "interrupt_response": true,
          "prefix_padding_ms": 0,
          "silence_duration_ms": 0,
          "threshold": 0
        }
      },
      "output": {
        "format": {
          "rate": 24000,
          "type": "audio/pcm"
        },
        "speed": 0.25,
        "voice": "ash"
      }
    },
    "expires_at": 0,
    "include": [
      "item.input_audio_transcription.logprobs"
    ],
    "instructions": "instructions",
    "max_output_tokens": "inf",
    "model": "gpt-realtime",
    "output_modalities": [
      "text"
    ],
    "prompt": {
      "id": "id",
      "variables": {
        "foo": "string"
      },
      "version": "version"
    },
    "reasoning": {
      "effort": "minimal"
    },
    "tool_choice": "none",
    "tools": [
      {
        "description": "description",
        "name": "name",
        "parameters": {},
        "type": "function"
      }
    ],
    "tracing": "auto",
    "truncation": "auto"
  },
  "value": "value"
}

Domain Types

Realtime Session Create Response

class RealtimeSessionCreateResponse

A Realtime session configuration object.
- id: String
  
  Unique identifier for the session that looks like sess_1234567890abcdef.
- object: :"realtime.session"
  
  The object type. Always realtime.session.
  - :"realtime.session"
- type: :realtime
  
  The type of session to create. Always realtime for the Realtime API.
  - :realtime
- audio: Audio{ input, output}
  
  Configuration for input and output audio.
  - input: Input{ format_, noise_reduction, transcription, turn_detection}
    - format_: RealtimeAudioFormats
      
      The format of the input audio.
      - class AudioPCM
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
        
        type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
      - class AudioPCMU
        
        The G.711 μ-law format.
        
        type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
      - class AudioPCMA
        
        The G.711 A-law format.
        
        type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
    - noise_reduction: NoiseReduction{ type}
      
      Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
      - type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
    - transcription: AudioTranscription
      - delay: :minimal | :low | :medium | 2 more
        
        Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
        
        :minimal
        
        :low
        
        :medium
        
        :high
        
        :xhigh
      - language: String
        
        The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
      - model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        String = String
        
        Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        :"whisper-1"
        
        :"gpt-4o-mini-transcribe"
        
        :"gpt-4o-mini-transcribe-2025-12-15"
        
        :"gpt-4o-transcribe"
        
        :"gpt-4o-transcribe-diarize"
        
        :"gpt-realtime-whisper"
      - prompt: String
        
        An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
    - turn_detection: ServerVad{ type, create_response, idle_timeout_ms, 4 more} | SemanticVad{ type, create_response, eagerness, interrupt_response}
      
      Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
      
      Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
      
      Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
      
      For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
      - class ServerVad
        
        Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
        
        type: :server_vad
        
        Type of turn detection, server_vad to turn on simple Server VAD.
        
        :server_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        idle_timeout_ms: Integer
        
        Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
        
        The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
        
        An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        prefix_padding_ms: Integer
        
        Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
        
        silence_duration_ms: Integer
        
        Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
        
        threshold: Float
        
        Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
      - class SemanticVad
        
        Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
        
        type: :semantic_vad
        
        Type of turn detection, semantic_vad to turn on Semantic VAD.
        
        :semantic_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs.
        
        eagerness: :low | :medium | :high | :auto
        
        Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
        
        :low
        
        :medium
        
        :high
        
        :auto
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.
  - output: Output{ format_, speed, voice}
    - format_: RealtimeAudioFormats
      
      The format of the output audio.
    - speed: Float
      
      The speed of the model's spoken response as a multiple of the original speed. 1.0 is the default speed. 0.25 is the minimum speed. 1.5 is the maximum speed. This value can only be changed in between model turns, not while a response is in progress.
      
      This parameter is a post-processing adjustment to the audio after it is generated, it's also possible to prompt the model to speak faster or slower.
    - voice: String | :alloy | :ash | :ballad | 7 more
      
      The voice the model uses to respond. Voice cannot be changed during the session once the model has responded with audio at least once. Current voice options are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. We recommend marin and cedar for best quality.
      - String = String
      - Voice = :alloy | :ash | :ballad | 7 more
        
        The voice the model uses to respond. Voice cannot be changed during the session once the model has responded with audio at least once. Current voice options are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. We recommend marin and cedar for best quality.
        
        :alloy
        
        :ash
        
        :ballad
        
        :coral
        
        :echo
        
        :sage
        
        :shimmer
        
        :verse
        
        :marin
        
        :cedar
- expires_at: Integer
  
  Expiration timestamp for the session, in seconds since epoch.
- include: Array[:"item.input_audio_transcription.logprobs"]
  
  Additional fields to include in server outputs.
  
  item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
  - :"item.input_audio_transcription.logprobs"
- instructions: String
  
  The default system instructions (i.e. system message) prepended to model calls. This field allows the client to guide the model on desired responses. The model can be instructed on response content and format, (e.g. "be extremely succinct", "act friendly", "here are examples of good responses") and on audio behavior (e.g. "talk quickly", "inject emotion into your voice", "laugh frequently"). The instructions are not guaranteed to be followed by the model, but they provide guidance to the model on the desired behavior.
  
  Note that the server sets default instructions which will be used if this field is not set and are visible in the session.created event at the start of the session.
- max_output_tokens: Integer | :inf
  
  Maximum number of output tokens for a single assistant response, inclusive of tool calls. Provide an integer between 1 and 4096 to limit output tokens, or inf for the maximum available tokens for a given model. Defaults to inf.
  - Integer = Integer
  - MaxOutputTokens = :inf
    - :inf
- model: String | :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
  
  The Realtime model used for this session.
  - String = String
  - Model = :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
    
    The Realtime model used for this session.
    - :"gpt-realtime"
    - :"gpt-realtime-1.5"
    - :"gpt-realtime-2"
    - :"gpt-realtime-2025-08-28"
    - :"gpt-4o-realtime-preview"
    - :"gpt-4o-realtime-preview-2024-10-01"
    - :"gpt-4o-realtime-preview-2024-12-17"
    - :"gpt-4o-realtime-preview-2025-06-03"
    - :"gpt-4o-mini-realtime-preview"
    - :"gpt-4o-mini-realtime-preview-2024-12-17"
    - :"gpt-realtime-mini"
    - :"gpt-realtime-mini-2025-10-06"
    - :"gpt-realtime-mini-2025-12-15"
    - :"gpt-audio-1.5"
    - :"gpt-audio-mini"
    - :"gpt-audio-mini-2025-10-06"
    - :"gpt-audio-mini-2025-12-15"
- output_modalities: Array[:text | :audio]
  
  The set of modalities the model can respond with. It defaults to ["audio"], indicating that the model will respond with audio plus a transcript. ["text"] can be used to make the model respond with text only. It is not possible to request both text and audio at the same time.
  - :text
  - :audio
- prompt: ResponsePrompt
  
  Reference to a prompt template and its variables. Learn more.
  - id: String
    
    The unique identifier of the prompt template to use.
  - variables: Hash[Symbol, String | ResponseInputText | ResponseInputImage | ResponseInputFile]
    
    Optional map of values to substitute in for variables in your prompt. The substitution values can either be strings, or other Response input types like images or files.
    - String = String
    - class ResponseInputText
      
      A text input to the model.
      - text: String
        
        The text input to the model.
      - type: :input_text
        
        The type of the input item. Always input_text.
        
        :input_text
    - class ResponseInputImage
      
      An image input to the model. Learn about image inputs.
      - detail: :low | :high | :auto | :original
        
        The detail level of the image to be sent to the model. One of high, low, auto, or original. Defaults to auto.
        
        :low
        
        :high
        
        :auto
        
        :original
      - type: :input_image
        
        The type of the input item. Always input_image.
        
        :input_image
      - file_id: String
        
        The ID of the file to be sent to the model.
      - image_url: String
        
        The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.
    - class ResponseInputFile
      
      A file input to the model.
      - type: :input_file
        
        The type of the input item. Always input_file.
        
        :input_file
      - detail: :low | :high
        
        The detail level of the file to be sent to the model. Use low for the default rendering behavior, or high to render the file at higher quality. Defaults to low.
        
        :low
        
        :high
      - file_data: String
        
        The content of the file to be sent to the model.
      - file_id: String
        
        The ID of the file to be sent to the model.
      - file_url: String
        
        The URL of the file to be sent to the model.
      - filename: String
        
        The name of the file to be sent to the model.
  - version: String
    
    Optional version of the prompt template.
- reasoning: RealtimeReasoning
  
  Configuration for reasoning-capable Realtime models such as gpt-realtime-2.
  - effort: RealtimeReasoningEffort
    
    Constrains effort on reasoning for reasoning-capable Realtime models such as gpt-realtime-2.
    - :minimal
    - :low
    - :medium
    - :high
    - :xhigh
- tool_choice: ToolChoiceOptions | ToolChoiceFunction | ToolChoiceMcp
  
  How the model chooses tools. Provide one of the string modes or force a specific function/MCP tool.
  - ToolChoiceOptions = :none | :auto | :required
    
    Controls which (if any) tool is called by the model.
    
    none means the model will not call any tool and instead generates a message.
    
    auto means the model can pick between generating a message or calling one or more tools.
    
    required means the model must call one or more tools.
    - :none
    - :auto
    - :required
  - class ToolChoiceFunction
    
    Use this option to force the model to call a specific function.
    - name: String
      
      The name of the function to call.
    - type: :function
      
      For function calling, the type is always function.
      - :function
  - class ToolChoiceMcp
    
    Use this option to force the model to call a specific tool on a remote MCP server.
    - server_label: String
      
      The label of the MCP server to use.
    - type: :mcp
      
      For MCP tools, the type is always mcp.
      - :mcp
    - name: String
      
      The name of the tool to call on the server.
- tools: Array[RealtimeFunctionTool | McpTool{ server_label, type, allowed_tools, 8 more}]
  
  Tools available to the model.
  - class RealtimeFunctionTool
    - description: String
      
      The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything).
    - name: String
      
      The name of the function.
    - parameters: untyped
      
      Parameters of the function in JSON Schema.
    - type: :function
      
      The type of the tool, i.e. function.
      - :function
  - class McpTool
    
    Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.
    - server_label: String
      
      A label for this MCP server, used to identify it in tool calls.
    - type: :mcp
      
      The type of the MCP tool. Always mcp.
      - :mcp
    - allowed_tools: Array[String] | McpToolFilter{ read_only, tool_names}
      
      List of allowed tool names or a filter object.
      - McpAllowedTools = Array[String]
        
        A string array of allowed tool names
      - class McpToolFilter
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
    - authorization: String
      
      An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.
    - connector_id: :connector_dropbox | :connector_gmail | :connector_googlecalendar | 5 more
      
      Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here.
      
      Currently supported connector_id values are:
      - Dropbox: connector_dropbox
      - Gmail: connector_gmail
      - Google Calendar: connector_googlecalendar
      - Google Drive: connector_googledrive
      - Microsoft Teams: connector_microsoftteams
      - Outlook Calendar: connector_outlookcalendar
      - Outlook Email: connector_outlookemail
      - SharePoint: connector_sharepoint
      - :connector_dropbox
      - :connector_gmail
      - :connector_googlecalendar
      - :connector_googledrive
      - :connector_microsoftteams
      - :connector_outlookcalendar
      - :connector_outlookemail
      - :connector_sharepoint
    - defer_loading: bool
      
      Whether this MCP tool is deferred and discovered via tool search.
    - headers: Hash[Symbol, String]
      
      Optional HTTP headers to send to the MCP server. Use for authentication or other purposes.
    - require_approval: McpToolApprovalFilter{ always, never} | :always | :never
      
      Specify which of the MCP server's tools require approval.
      - class McpToolApprovalFilter
        
        Specify which of the MCP server's tools require approval. Can be always, never, or a filter object associated with tools that require approval.
        
        always: Always{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        never: Never{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
      - McpToolApprovalSetting = :always | :never
        
        Specify a single approval policy for all tools. One of always or never. When set to always, all tools will require approval. When set to never, all tools will not require approval.
        
        :always
        
        :never
    - server_description: String
      
      Optional description of the MCP server, used to provide more context.
    - server_url: String
      
      The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided.
    - tunnel_id: String
      
      The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided.
- tracing: :auto | TracingConfiguration{ group_id, metadata, workflow_name}
  
  Realtime API can write session traces to the Traces Dashboard. Set to null to disable tracing. Once tracing is enabled for a session, the configuration cannot be modified.
  
  auto will create a trace for the session with default values for the workflow name, group id, and metadata.
  - Tracing = :auto
    
    Enables tracing and sets default values for tracing configuration options. Always auto.
    - :auto
  - class TracingConfiguration
    
    Granular configuration for tracing.
    - group_id: String
      
      The group id to attach to this trace to enable filtering and grouping in the Traces Dashboard.
    - metadata: untyped
      
      The arbitrary metadata to attach to this trace to enable filtering in the Traces Dashboard.
    - workflow_name: String
      
      The name of the workflow to attach to this trace. This is used to name the trace in the Traces Dashboard.
- truncation: RealtimeTruncation
  
  When the number of tokens in a conversation exceeds the model's input token limit, the conversation be truncated, meaning messages (starting from the oldest) will not be included in the model's context. A 32k context model with 4,096 max output tokens can only include 28,224 tokens in the context before truncation occurs.
  
  Clients can configure truncation behavior to truncate with a lower max token limit, which is an effective way to control token usage and cost.
  
  Truncation will reduce the number of cached tokens on the next turn (busting the cache), since messages are dropped from the beginning of the context. However, clients can also configure truncation to retain messages up to a fraction of the maximum context size, which will reduce the need for future truncations and thus improve the cache rate.
  
  Truncation can be disabled entirely, which means the server will never truncate but would instead return an error if the conversation exceeds the model's input token limit.
  - RealtimeTruncationStrategy = :auto | :disabled
    
    The truncation strategy to use for the session. auto is the default truncation strategy. disabled will disable truncation and emit errors when the conversation exceeds the input token limit.
    - :auto
    - :disabled
  - class RealtimeTruncationRetentionRatio
    
    Retain a fraction of the conversation tokens when the conversation exceeds the input token limit. This allows you to amortize truncations across multiple turns, which can help improve cached token usage.
    - retention_ratio: Float
      
      Fraction of post-instruction conversation tokens to retain (0.0 - 1.0) when the conversation exceeds the input token limit. Setting this to 0.8 means that messages will be dropped until 80% of the maximum allowed tokens are used. This helps reduce the frequency of truncations and improve cache rates.
    - type: :retention_ratio
      
      Use retention ratio truncation.
      - :retention_ratio
    - token_limits: TokenLimits{ post_instructions}
      
      Optional custom token limits for this truncation strategy. If not provided, the model's default token limits will be used.
      - post_instructions: Integer
        
        Maximum tokens allowed in the conversation after instructions (which including tool definitions). For example, setting this to 5,000 would mean that truncation would occur when the conversation exceeds 5,000 tokens after instructions. This cannot be higher than the model's context window size minus the maximum output tokens.

Realtime Transcription Session Create Response

class RealtimeTranscriptionSessionCreateResponse

A Realtime transcription session configuration object.
- id: String
  
  Unique identifier for the session that looks like sess_1234567890abcdef.
- object: String
  
  The object type. Always realtime.transcription_session.
- type: :transcription
  
  The type of session. Always transcription for transcription sessions.
  - :transcription
- audio: Audio{ input}
  
  Configuration for input audio for the session.
  - input: Input{ format_, noise_reduction, transcription, turn_detection}
    - format_: RealtimeAudioFormats
      
      The PCM audio format. Only a 24kHz sample rate is supported.
      - class AudioPCM
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
        
        type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
      - class AudioPCMU
        
        The G.711 μ-law format.
        
        type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
      - class AudioPCMA
        
        The G.711 A-law format.
        
        type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
    - noise_reduction: NoiseReduction{ type}
      
      Configuration for input audio noise reduction.
      - type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
    - transcription: AudioTranscription
      - delay: :minimal | :low | :medium | 2 more
        
        Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
        
        :minimal
        
        :low
        
        :medium
        
        :high
        
        :xhigh
      - language: String
        
        The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
      - model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        String = String
        
        Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        :"whisper-1"
        
        :"gpt-4o-mini-transcribe"
        
        :"gpt-4o-mini-transcribe-2025-12-15"
        
        :"gpt-4o-transcribe"
        
        :"gpt-4o-transcribe-diarize"
        
        :"gpt-realtime-whisper"
      - prompt: String
        
        An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
    - turn_detection: RealtimeTranscriptionSessionTurnDetection
      
      Configuration for turn detection. Can be set to null to turn off. Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech. For gpt-realtime-whisper, this must be null; VAD is not supported.
      - prefix_padding_ms: Integer
        
        Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
      - silence_duration_ms: Integer
        
        Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
      - threshold: Float
        
        Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
      - type: String
        
        Type of turn detection, only server_vad is currently supported.
- expires_at: Integer
  
  Expiration timestamp for the session, in seconds since epoch.
- include: Array[:"item.input_audio_transcription.logprobs"]
  
  Additional fields to include in server outputs.
  - item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
  - :"item.input_audio_transcription.logprobs"

Realtime Transcription Session Turn Detection

class RealtimeTranscriptionSessionTurnDetection

Configuration for turn detection. Can be set to null to turn off. Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech. For gpt-realtime-whisper, this must be null; VAD is not supported.
- prefix_padding_ms: Integer
  
  Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
- silence_duration_ms: Integer
  
  Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
- threshold: Float
  
  Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
- type: String
  
  Type of turn detection, only server_vad is currently supported.

Client Secret Create Response

class ClientSecretCreateResponse

Response from creating a session and client secret for the Realtime API.
- expires_at: Integer
  
  Expiration timestamp for the client secret, in seconds since epoch.
- session: RealtimeSessionCreateResponse | RealtimeTranscriptionSessionCreateResponse
  
  The session configuration for either a realtime or transcription session.
  - class RealtimeSessionCreateResponse
    
    A Realtime session configuration object.
    - id: String
      
      Unique identifier for the session that looks like sess_1234567890abcdef.
    - object: :"realtime.session"
      
      The object type. Always realtime.session.
      - :"realtime.session"
    - type: :realtime
      
      The type of session to create. Always realtime for the Realtime API.
      - :realtime
    - audio: Audio{ input, output}
      
      Configuration for input and output audio.
      - input: Input{ format_, noise_reduction, transcription, turn_detection}
        
        format_: RealtimeAudioFormats
        
        The format of the input audio.
        
        class AudioPCM
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
        
        type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
        
        class AudioPCMU
        
        The G.711 μ-law format.
        
        type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
        
        class AudioPCMA
        
        The G.711 A-law format.
        
        type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
        
        noise_reduction: NoiseReduction{ type}
        
        Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        :near_field
        
        :far_field
        
        transcription: AudioTranscription
        
        delay: :minimal | :low | :medium | 2 more
        
        Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
        
        :minimal
        
        :low
        
        :medium
        
        :high
        
        :xhigh
        
        language: String
        
        The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
        
        model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        String = String
        
        Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        :"whisper-1"
        
        :"gpt-4o-mini-transcribe"
        
        :"gpt-4o-mini-transcribe-2025-12-15"
        
        :"gpt-4o-transcribe"
        
        :"gpt-4o-transcribe-diarize"
        
        :"gpt-realtime-whisper"
        
        prompt: String
        
        An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
        
        turn_detection: ServerVad{ type, create_response, idle_timeout_ms, 4 more} | SemanticVad{ type, create_response, eagerness, interrupt_response}
        
        Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
        
        Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
        
        Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
        
        For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
        
        class ServerVad
        
        Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
        
        type: :server_vad
        
        Type of turn detection, server_vad to turn on simple Server VAD.
        
        :server_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        idle_timeout_ms: Integer
        
        Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
        
        The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
        
        An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
        
        prefix_padding_ms: Integer
        
        Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
        
        silence_duration_ms: Integer
        
        Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
        
        threshold: Float
        
        Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
        
        class SemanticVad
        
        Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
        
        type: :semantic_vad
        
        Type of turn detection, semantic_vad to turn on Semantic VAD.
        
        :semantic_vad
        
        create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs.
        
        eagerness: :low | :medium | :high | :auto
        
        Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
        
        :low
        
        :medium
        
        :high
        
        :auto
        
        interrupt_response: bool
        
        Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.
      - output: Output{ format_, speed, voice}
        
        format_: RealtimeAudioFormats
        
        The format of the output audio.
        
        speed: Float
        
        The speed of the model's spoken response as a multiple of the original speed. 1.0 is the default speed. 0.25 is the minimum speed. 1.5 is the maximum speed. This value can only be changed in between model turns, not while a response is in progress.
        
        This parameter is a post-processing adjustment to the audio after it is generated, it's also possible to prompt the model to speak faster or slower.
        
        voice: String | :alloy | :ash | :ballad | 7 more
        
        The voice the model uses to respond. Voice cannot be changed during the session once the model has responded with audio at least once. Current voice options are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. We recommend marin and cedar for best quality.
        
        String = String
        
        Voice = :alloy | :ash | :ballad | 7 more
        
        The voice the model uses to respond. Voice cannot be changed during the session once the model has responded with audio at least once. Current voice options are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. We recommend marin and cedar for best quality.
        
        :alloy
        
        :ash
        
        :ballad
        
        :coral
        
        :echo
        
        :sage
        
        :shimmer
        
        :verse
        
        :marin
        
        :cedar
    - expires_at: Integer
      
      Expiration timestamp for the session, in seconds since epoch.
    - include: Array[:"item.input_audio_transcription.logprobs"]
      
      Additional fields to include in server outputs.
      
      item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
      - :"item.input_audio_transcription.logprobs"
    - instructions: String
      
      The default system instructions (i.e. system message) prepended to model calls. This field allows the client to guide the model on desired responses. The model can be instructed on response content and format, (e.g. "be extremely succinct", "act friendly", "here are examples of good responses") and on audio behavior (e.g. "talk quickly", "inject emotion into your voice", "laugh frequently"). The instructions are not guaranteed to be followed by the model, but they provide guidance to the model on the desired behavior.
      
      Note that the server sets default instructions which will be used if this field is not set and are visible in the session.created event at the start of the session.
    - max_output_tokens: Integer | :inf
      
      Maximum number of output tokens for a single assistant response, inclusive of tool calls. Provide an integer between 1 and 4096 to limit output tokens, or inf for the maximum available tokens for a given model. Defaults to inf.
      - Integer = Integer
      - MaxOutputTokens = :inf
        
        :inf
    - model: String | :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
      
      The Realtime model used for this session.
      - String = String
      - Model = :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
        
        The Realtime model used for this session.
        
        :"gpt-realtime"
        
        :"gpt-realtime-1.5"
        
        :"gpt-realtime-2"
        
        :"gpt-realtime-2025-08-28"
        
        :"gpt-4o-realtime-preview"
        
        :"gpt-4o-realtime-preview-2024-10-01"
        
        :"gpt-4o-realtime-preview-2024-12-17"
        
        :"gpt-4o-realtime-preview-2025-06-03"
        
        :"gpt-4o-mini-realtime-preview"
        
        :"gpt-4o-mini-realtime-preview-2024-12-17"
        
        :"gpt-realtime-mini"
        
        :"gpt-realtime-mini-2025-10-06"
        
        :"gpt-realtime-mini-2025-12-15"
        
        :"gpt-audio-1.5"
        
        :"gpt-audio-mini"
        
        :"gpt-audio-mini-2025-10-06"
        
        :"gpt-audio-mini-2025-12-15"
    - output_modalities: Array[:text | :audio]
      
      The set of modalities the model can respond with. It defaults to ["audio"], indicating that the model will respond with audio plus a transcript. ["text"] can be used to make the model respond with text only. It is not possible to request both text and audio at the same time.
      - :text
      - :audio
    - prompt: ResponsePrompt
      
      Reference to a prompt template and its variables. Learn more.
      - id: String
        
        The unique identifier of the prompt template to use.
      - variables: Hash[Symbol, String | ResponseInputText | ResponseInputImage | ResponseInputFile]
        
        Optional map of values to substitute in for variables in your prompt. The substitution values can either be strings, or other Response input types like images or files.
        
        String = String
        
        class ResponseInputText
        
        A text input to the model.
        
        text: String
        
        The text input to the model.
        
        type: :input_text
        
        The type of the input item. Always input_text.
        
        :input_text
        
        class ResponseInputImage
        
        An image input to the model. Learn about image inputs.
        
        detail: :low | :high | :auto | :original
        
        The detail level of the image to be sent to the model. One of high, low, auto, or original. Defaults to auto.
        
        :low
        
        :high
        
        :auto
        
        :original
        
        type: :input_image
        
        The type of the input item. Always input_image.
        
        :input_image
        
        file_id: String
        
        The ID of the file to be sent to the model.
        
        image_url: String
        
        The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.
        
        class ResponseInputFile
        
        A file input to the model.
        
        type: :input_file
        
        The type of the input item. Always input_file.
        
        :input_file
        
        detail: :low | :high
        
        The detail level of the file to be sent to the model. Use low for the default rendering behavior, or high to render the file at higher quality. Defaults to low.
        
        :low
        
        :high
        
        file_data: String
        
        The content of the file to be sent to the model.
        
        file_id: String
        
        The ID of the file to be sent to the model.
        
        file_url: String
        
        The URL of the file to be sent to the model.
        
        filename: String
        
        The name of the file to be sent to the model.
      - version: String
        
        Optional version of the prompt template.
    - reasoning: RealtimeReasoning
      
      Configuration for reasoning-capable Realtime models such as gpt-realtime-2.
      - effort: RealtimeReasoningEffort
        
        Constrains effort on reasoning for reasoning-capable Realtime models such as gpt-realtime-2.
        
        :minimal
        
        :low
        
        :medium
        
        :high
        
        :xhigh
    - tool_choice: ToolChoiceOptions | ToolChoiceFunction | ToolChoiceMcp
      
      How the model chooses tools. Provide one of the string modes or force a specific function/MCP tool.
      - ToolChoiceOptions = :none | :auto | :required
        
        Controls which (if any) tool is called by the model.
        
        none means the model will not call any tool and instead generates a message.
        
        auto means the model can pick between generating a message or calling one or more tools.
        
        required means the model must call one or more tools.
        
        :none
        
        :auto
        
        :required
      - class ToolChoiceFunction
        
        Use this option to force the model to call a specific function.
        
        name: String
        
        The name of the function to call.
        
        type: :function
        
        For function calling, the type is always function.
        
        :function
      - class ToolChoiceMcp
        
        Use this option to force the model to call a specific tool on a remote MCP server.
        
        server_label: String
        
        The label of the MCP server to use.
        
        type: :mcp
        
        For MCP tools, the type is always mcp.
        
        :mcp
        
        name: String
        
        The name of the tool to call on the server.
    - tools: Array[RealtimeFunctionTool | McpTool{ server_label, type, allowed_tools, 8 more}]
      
      Tools available to the model.
      - class RealtimeFunctionTool
        
        description: String
        
        The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything).
        
        name: String
        
        The name of the function.
        
        parameters: untyped
        
        Parameters of the function in JSON Schema.
        
        type: :function
        
        The type of the tool, i.e. function.
        
        :function
      - class McpTool
        
        Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.
        
        server_label: String
        
        A label for this MCP server, used to identify it in tool calls.
        
        type: :mcp
        
        The type of the MCP tool. Always mcp.
        
        :mcp
        
        allowed_tools: Array[String] | McpToolFilter{ read_only, tool_names}
        
        List of allowed tool names or a filter object.
        
        McpAllowedTools = Array[String]
        
        A string array of allowed tool names
        
        class McpToolFilter
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        authorization: String
        
        An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.
        
        connector_id: :connector_dropbox | :connector_gmail | :connector_googlecalendar | 5 more
        
        Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here.
        
        Currently supported connector_id values are:
        
        Dropbox: connector_dropbox
        
        Gmail: connector_gmail
        
        Google Calendar: connector_googlecalendar
        
        Google Drive: connector_googledrive
        
        Microsoft Teams: connector_microsoftteams
        
        Outlook Calendar: connector_outlookcalendar
        
        Outlook Email: connector_outlookemail
        
        SharePoint: connector_sharepoint
        
        :connector_dropbox
        
        :connector_gmail
        
        :connector_googlecalendar
        
        :connector_googledrive
        
        :connector_microsoftteams
        
        :connector_outlookcalendar
        
        :connector_outlookemail
        
        :connector_sharepoint
        
        defer_loading: bool
        
        Whether this MCP tool is deferred and discovered via tool search.
        
        headers: Hash[Symbol, String]
        
        Optional HTTP headers to send to the MCP server. Use for authentication or other purposes.
        
        require_approval: McpToolApprovalFilter{ always, never} | :always | :never
        
        Specify which of the MCP server's tools require approval.
        
        class McpToolApprovalFilter
        
        Specify which of the MCP server's tools require approval. Can be always, never, or a filter object associated with tools that require approval.
        
        always: Always{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        never: Never{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
        
        McpToolApprovalSetting = :always | :never
        
        Specify a single approval policy for all tools. One of always or never. When set to always, all tools will require approval. When set to never, all tools will not require approval.
        
        :always
        
        :never
        
        server_description: String
        
        Optional description of the MCP server, used to provide more context.
        
        server_url: String
        
        The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided.
        
        tunnel_id: String
        
        The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided.
    - tracing: :auto | TracingConfiguration{ group_id, metadata, workflow_name}
      
      Realtime API can write session traces to the Traces Dashboard. Set to null to disable tracing. Once tracing is enabled for a session, the configuration cannot be modified.
      
      auto will create a trace for the session with default values for the workflow name, group id, and metadata.
      - Tracing = :auto
        
        Enables tracing and sets default values for tracing configuration options. Always auto.
        
        :auto
      - class TracingConfiguration
        
        Granular configuration for tracing.
        
        group_id: String
        
        The group id to attach to this trace to enable filtering and grouping in the Traces Dashboard.
        
        metadata: untyped
        
        The arbitrary metadata to attach to this trace to enable filtering in the Traces Dashboard.
        
        workflow_name: String
        
        The name of the workflow to attach to this trace. This is used to name the trace in the Traces Dashboard.
    - truncation: RealtimeTruncation
      
      When the number of tokens in a conversation exceeds the model's input token limit, the conversation be truncated, meaning messages (starting from the oldest) will not be included in the model's context. A 32k context model with 4,096 max output tokens can only include 28,224 tokens in the context before truncation occurs.
      
      Clients can configure truncation behavior to truncate with a lower max token limit, which is an effective way to control token usage and cost.
      
      Truncation will reduce the number of cached tokens on the next turn (busting the cache), since messages are dropped from the beginning of the context. However, clients can also configure truncation to retain messages up to a fraction of the maximum context size, which will reduce the need for future truncations and thus improve the cache rate.
      
      Truncation can be disabled entirely, which means the server will never truncate but would instead return an error if the conversation exceeds the model's input token limit.
      - RealtimeTruncationStrategy = :auto | :disabled
        
        The truncation strategy to use for the session. auto is the default truncation strategy. disabled will disable truncation and emit errors when the conversation exceeds the input token limit.
        
        :auto
        
        :disabled
      - class RealtimeTruncationRetentionRatio
        
        Retain a fraction of the conversation tokens when the conversation exceeds the input token limit. This allows you to amortize truncations across multiple turns, which can help improve cached token usage.
        
        retention_ratio: Float
        
        Fraction of post-instruction conversation tokens to retain (0.0 - 1.0) when the conversation exceeds the input token limit. Setting this to 0.8 means that messages will be dropped until 80% of the maximum allowed tokens are used. This helps reduce the frequency of truncations and improve cache rates.
        
        type: :retention_ratio
        
        Use retention ratio truncation.
        
        :retention_ratio
        
        token_limits: TokenLimits{ post_instructions}
        
        Optional custom token limits for this truncation strategy. If not provided, the model's default token limits will be used.
        
        post_instructions: Integer
        
        Maximum tokens allowed in the conversation after instructions (which including tool definitions). For example, setting this to 5,000 would mean that truncation would occur when the conversation exceeds 5,000 tokens after instructions. This cannot be higher than the model's context window size minus the maximum output tokens.
  - class RealtimeTranscriptionSessionCreateResponse
    
    A Realtime transcription session configuration object.
    - id: String
      
      Unique identifier for the session that looks like sess_1234567890abcdef.
    - object: String
      
      The object type. Always realtime.transcription_session.
    - type: :transcription
      
      The type of session. Always transcription for transcription sessions.
      - :transcription
    - audio: Audio{ input}
      
      Configuration for input audio for the session.
      - input: Input{ format_, noise_reduction, transcription, turn_detection}
        
        format_: RealtimeAudioFormats
        
        The PCM audio format. Only a 24kHz sample rate is supported.
        
        noise_reduction: NoiseReduction{ type}
        
        Configuration for input audio noise reduction.
        
        type: NoiseReductionType
        
        Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
        
        transcription: AudioTranscription
        
        turn_detection: RealtimeTranscriptionSessionTurnDetection
        
        Configuration for turn detection. Can be set to null to turn off. Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech. For gpt-realtime-whisper, this must be null; VAD is not supported.
        
        prefix_padding_ms: Integer
        
        Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
        
        silence_duration_ms: Integer
        
        Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
        
        threshold: Float
        
        Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
        
        type: String
        
        Type of turn detection, only server_vad is currently supported.
    - expires_at: Integer
      
      Expiration timestamp for the session, in seconds since epoch.
    - include: Array[:"item.input_audio_transcription.logprobs"]
      
      Additional fields to include in server outputs.
      - item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
      - :"item.input_audio_transcription.logprobs"
- value: String
  
  The generated client secret value.

Calls

Accept call

realtime.calls.accept(call_id, **kwargs) -> void

post /realtime/calls/{call_id}/accept

Accept an incoming SIP call and configure the realtime session that will handle it.

Parameters

call_id: String
type: :realtime

The type of session to create. Always realtime for the Realtime API.
- :realtime
audio: RealtimeAudioConfig

Configuration for input and output audio.
- input: RealtimeAudioConfigInput
  - format_: RealtimeAudioFormats
    
    The format of the input audio.
    - class AudioPCM
      
      The PCM audio format. Only a 24kHz sample rate is supported.
      - rate: 24000
        
        The sample rate of the audio. Always 24000.
        
        24000
      - type: :"audio/pcm"
        
        The audio format. Always audio/pcm.
        
        :"audio/pcm"
    - class AudioPCMU
      
      The G.711 μ-law format.
      - type: :"audio/pcmu"
        
        The audio format. Always audio/pcmu.
        
        :"audio/pcmu"
    - class AudioPCMA
      
      The G.711 A-law format.
      - type: :"audio/pcma"
        
        The audio format. Always audio/pcma.
        
        :"audio/pcma"
  - noise_reduction: NoiseReduction{ type}
    
    Configuration for input audio noise reduction. This can be set to null to turn off. Noise reduction filters audio added to the input audio buffer before it is sent to VAD and the model. Filtering the audio can improve VAD and turn detection accuracy (reducing false positives) and model performance by improving perception of the input audio.
    - type: NoiseReductionType
      
      Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.
      - :near_field
      - :far_field
  - transcription: AudioTranscription
    
    Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.
    - delay: :minimal | :low | :medium | 2 more
      
      Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
      - :minimal
      - :low
      - :medium
      - :high
      - :xhigh
    - language: String
      
      The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.
    - model: String | :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
      
      The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
      - String = String
      - Model = :"whisper-1" | :"gpt-4o-mini-transcribe" | :"gpt-4o-mini-transcribe-2025-12-15" | 3 more
        
        The model to use for transcription. Current options are whisper-1, gpt-4o-mini-transcribe, gpt-4o-mini-transcribe-2025-12-15, gpt-4o-transcribe, gpt-4o-transcribe-diarize, and gpt-realtime-whisper. Use gpt-4o-transcribe-diarize when you need diarization with speaker labels.
        
        :"whisper-1"
        
        :"gpt-4o-mini-transcribe"
        
        :"gpt-4o-mini-transcribe-2025-12-15"
        
        :"gpt-4o-transcribe"
        
        :"gpt-4o-transcribe-diarize"
        
        :"gpt-realtime-whisper"
    - prompt: String
      
      An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
  - turn_detection: RealtimeAudioInputTurnDetection
    
    Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.
    
    Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.
    
    Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.
    
    For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.
    - class ServerVad
      
      Server-side voice activity detection (VAD) which flips on when user speech is detected and off after a period of silence.
      - type: :server_vad
        
        Type of turn detection, server_vad to turn on simple Server VAD.
        
        :server_vad
      - create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs. If interrupt_response is set to false this may fail to create a response if the model is already responding.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
      - idle_timeout_ms: Integer
        
        Optional timeout after which a model response will be triggered automatically. This is useful for situations in which a long pause from the user is unexpected, such as a phone call. The model will effectively prompt the user to continue the conversation based on the current context.
        
        The timeout value will be applied after the last model response's audio has finished playing, i.e. it's set to the response.done time plus audio playback duration.
        
        An input_audio_buffer.timeout_triggered event (plus events associated with the Response) will be emitted when the timeout is reached. Idle timeout is currently only supported for server_vad mode.
      - interrupt_response: bool
        
        Whether or not to automatically interrupt (cancel) any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs. If true then the response will be cancelled, otherwise it will continue until complete.
        
        If both create_response and interrupt_response are set to false, the model will never respond automatically but VAD events will still be emitted.
      - prefix_padding_ms: Integer
        
        Used only for server_vad mode. Amount of audio to include before the VAD detected speech (in milliseconds). Defaults to 300ms.
      - silence_duration_ms: Integer
        
        Used only for server_vad mode. Duration of silence to detect speech stop (in milliseconds). Defaults to 500ms. With shorter values the model will respond more quickly, but may jump in on short pauses from the user.
      - threshold: Float
        
        Used only for server_vad mode. Activation threshold for VAD (0.0 to 1.0), this defaults to 0.5. A higher threshold will require louder audio to activate the model, and thus might perform better in noisy environments.
    - class SemanticVad
      
      Server-side semantic turn detection which uses a model to determine when the user has finished speaking.
      - type: :semantic_vad
        
        Type of turn detection, semantic_vad to turn on Semantic VAD.
        
        :semantic_vad
      - create_response: bool
        
        Whether or not to automatically generate a response when a VAD stop event occurs.
      - eagerness: :low | :medium | :high | :auto
        
        Used only for semantic_vad mode. The eagerness of the model to respond. low will wait longer for the user to continue speaking, high will respond more quickly. auto is the default and is equivalent to medium. low, medium, and high have max timeouts of 8s, 4s, and 2s respectively.
        
        :low
        
        :medium
        
        :high
        
        :auto
      - interrupt_response: bool
        
        Whether or not to automatically interrupt any ongoing response with output to the default conversation (i.e. conversation of auto) when a VAD start event occurs.
- output: RealtimeAudioConfigOutput
  - format_: RealtimeAudioFormats
    
    The format of the output audio.
  - speed: Float
    
    The speed of the model's spoken response as a multiple of the original speed. 1.0 is the default speed. 0.25 is the minimum speed. 1.5 is the maximum speed. This value can only be changed in between model turns, not while a response is in progress.
    
    This parameter is a post-processing adjustment to the audio after it is generated, it's also possible to prompt the model to speak faster or slower.
  - voice: String | :alloy | :ash | :ballad | 7 more | ID{ id}
    
    The voice the model uses to respond. Supported built-in voices are alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, and cedar. You may also provide a custom voice object with an id, for example { "id": "voice_1234" }. Voice cannot be changed during the session once the model has responded with audio at least once. We recommend marin and cedar for best quality.
    - String = String
    - Voice = :alloy | :ash | :ballad | 7 more
      - :alloy
      - :ash
      - :ballad
      - :coral
      - :echo
      - :sage
      - :shimmer
      - :verse
      - :marin
      - :cedar
    - class ID
      
      Custom voice reference.
      - id: String
        
        The custom voice ID, e.g. voice_1234.
include: Array[:"item.input_audio_transcription.logprobs"]

Additional fields to include in server outputs.

item.input_audio_transcription.logprobs: Include logprobs for input audio transcription.
- :"item.input_audio_transcription.logprobs"
instructions: String

The default system instructions (i.e. system message) prepended to model calls. This field allows the client to guide the model on desired responses. The model can be instructed on response content and format, (e.g. "be extremely succinct", "act friendly", "here are examples of good responses") and on audio behavior (e.g. "talk quickly", "inject emotion into your voice", "laugh frequently"). The instructions are not guaranteed to be followed by the model, but they provide guidance to the model on the desired behavior.

Note that the server sets default instructions which will be used if this field is not set and are visible in the session.created event at the start of the session.
max_output_tokens: Integer | :inf

Maximum number of output tokens for a single assistant response, inclusive of tool calls. Provide an integer between 1 and 4096 to limit output tokens, or inf for the maximum available tokens for a given model. Defaults to inf.
- Integer = Integer
- MaxOutputTokens = :inf
  - :inf
model: String | :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more

The Realtime model used for this session.
- String = String
- Model = :"gpt-realtime" | :"gpt-realtime-1.5" | :"gpt-realtime-2" | 14 more
  
  The Realtime model used for this session.
  - :"gpt-realtime"
  - :"gpt-realtime-1.5"
  - :"gpt-realtime-2"
  - :"gpt-realtime-2025-08-28"
  - :"gpt-4o-realtime-preview"
  - :"gpt-4o-realtime-preview-2024-10-01"
  - :"gpt-4o-realtime-preview-2024-12-17"
  - :"gpt-4o-realtime-preview-2025-06-03"
  - :"gpt-4o-mini-realtime-preview"
  - :"gpt-4o-mini-realtime-preview-2024-12-17"
  - :"gpt-realtime-mini"
  - :"gpt-realtime-mini-2025-10-06"
  - :"gpt-realtime-mini-2025-12-15"
  - :"gpt-audio-1.5"
  - :"gpt-audio-mini"
  - :"gpt-audio-mini-2025-10-06"
  - :"gpt-audio-mini-2025-12-15"
output_modalities: Array[:text | :audio]

The set of modalities the model can respond with. It defaults to ["audio"], indicating that the model will respond with audio plus a transcript. ["text"] can be used to make the model respond with text only. It is not possible to request both text and audio at the same time.
- :text
- :audio
parallel_tool_calls: bool

Whether the model may call multiple tools in parallel. Only supported by reasoning Realtime models such as gpt-realtime-2.
prompt: ResponsePrompt

Reference to a prompt template and its variables. Learn more.
- id: String
  
  The unique identifier of the prompt template to use.
- variables: Hash[Symbol, String | ResponseInputText | ResponseInputImage | ResponseInputFile]
  
  Optional map of values to substitute in for variables in your prompt. The substitution values can either be strings, or other Response input types like images or files.
  - String = String
  - class ResponseInputText
    
    A text input to the model.
    - text: String
      
      The text input to the model.
    - type: :input_text
      
      The type of the input item. Always input_text.
      - :input_text
  - class ResponseInputImage
    
    An image input to the model. Learn about image inputs.
    - detail: :low | :high | :auto | :original
      
      The detail level of the image to be sent to the model. One of high, low, auto, or original. Defaults to auto.
      - :low
      - :high
      - :auto
      - :original
    - type: :input_image
      
      The type of the input item. Always input_image.
      - :input_image
    - file_id: String
      
      The ID of the file to be sent to the model.
    - image_url: String
      
      The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.
  - class ResponseInputFile
    
    A file input to the model.
    - type: :input_file
      
      The type of the input item. Always input_file.
      - :input_file
    - detail: :low | :high
      
      The detail level of the file to be sent to the model. Use low for the default rendering behavior, or high to render the file at higher quality. Defaults to low.
      - :low
      - :high
    - file_data: String
      
      The content of the file to be sent to the model.
    - file_id: String
      
      The ID of the file to be sent to the model.
    - file_url: String
      
      The URL of the file to be sent to the model.
    - filename: String
      
      The name of the file to be sent to the model.
- version: String
  
  Optional version of the prompt template.
reasoning: RealtimeReasoning

Configuration for reasoning-capable Realtime models such as gpt-realtime-2.
- effort: RealtimeReasoningEffort
  
  Constrains effort on reasoning for reasoning-capable Realtime models such as gpt-realtime-2.
  - :minimal
  - :low
  - :medium
  - :high
  - :xhigh
tool_choice: RealtimeToolChoiceConfig

How the model chooses tools. Provide one of the string modes or force a specific function/MCP tool.
- ToolChoiceOptions = :none | :auto | :required
  
  Controls which (if any) tool is called by the model.
  
  none means the model will not call any tool and instead generates a message.
  
  auto means the model can pick between generating a message or calling one or more tools.
  
  required means the model must call one or more tools.
  - :none
  - :auto
  - :required
- class ToolChoiceFunction
  
  Use this option to force the model to call a specific function.
  - name: String
    
    The name of the function to call.
  - type: :function
    
    For function calling, the type is always function.
    - :function
- class ToolChoiceMcp
  
  Use this option to force the model to call a specific tool on a remote MCP server.
  - server_label: String
    
    The label of the MCP server to use.
  - type: :mcp
    
    For MCP tools, the type is always mcp.
    - :mcp
  - name: String
    
    The name of the tool to call on the server.
tools: RealtimeToolsConfig

Tools available to the model.
- class RealtimeFunctionTool
  - description: String
    
    The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything).
  - name: String
    
    The name of the function.
  - parameters: untyped
    
    Parameters of the function in JSON Schema.
  - type: :function
    
    The type of the tool, i.e. function.
    - :function
- class Mcp
  
  Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.
  - server_label: String
    
    A label for this MCP server, used to identify it in tool calls.
  - type: :mcp
    
    The type of the MCP tool. Always mcp.
    - :mcp
  - allowed_tools: Array[String] | McpToolFilter{ read_only, tool_names}
    
    List of allowed tool names or a filter object.
    - McpAllowedTools = Array[String]
      
      A string array of allowed tool names
    - class McpToolFilter
      
      A filter object to specify which tools are allowed.
      - read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
      - tool_names: Array[String]
        
        List of allowed tool names.
  - authorization: String
    
    An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.
  - connector_id: :connector_dropbox | :connector_gmail | :connector_googlecalendar | 5 more
    
    Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here.
    
    Currently supported connector_id values are:
    - Dropbox: connector_dropbox
    - Gmail: connector_gmail
    - Google Calendar: connector_googlecalendar
    - Google Drive: connector_googledrive
    - Microsoft Teams: connector_microsoftteams
    - Outlook Calendar: connector_outlookcalendar
    - Outlook Email: connector_outlookemail
    - SharePoint: connector_sharepoint
    - :connector_dropbox
    - :connector_gmail
    - :connector_googlecalendar
    - :connector_googledrive
    - :connector_microsoftteams
    - :connector_outlookcalendar
    - :connector_outlookemail
    - :connector_sharepoint
  - defer_loading: bool
    
    Whether this MCP tool is deferred and discovered via tool search.
  - headers: Hash[Symbol, String]
    
    Optional HTTP headers to send to the MCP server. Use for authentication or other purposes.
  - require_approval: McpToolApprovalFilter{ always, never} | :always | :never
    
    Specify which of the MCP server's tools require approval.
    - class McpToolApprovalFilter
      
      Specify which of the MCP server's tools require approval. Can be always, never, or a filter object associated with tools that require approval.
      - always: Always{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
      - never: Never{ read_only, tool_names}
        
        A filter object to specify which tools are allowed.
        
        read_only: bool
        
        Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.
        
        tool_names: Array[String]
        
        List of allowed tool names.
    - McpToolApprovalSetting = :always | :never
      
      Specify a single approval policy for all tools. One of always or never. When set to always, all tools will require approval. When set to never, all tools will not require approval.
      - :always
      - :never
  - server_description: String
    
    Optional description of the MCP server, used to provide more context.
  - server_url: String
    
    The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided.
  - tunnel_id: String
    
    The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided.
tracing: RealtimeTracingConfig

Realtime API can write session traces to the Traces Dashboard. Set to null to disable tracing. Once tracing is enabled for a session, the configuration cannot be modified.

auto will create a trace for the session with default values for the workflow name, group id, and metadata.
- RealtimeTracingConfig = :auto
  
  Enables tracing and sets default values for tracing configuration options. Always auto.
  - :auto
- class TracingConfiguration
  
  Granular configuration for tracing.
  - group_id: String
    
    The group id to attach to this trace to enable filtering and grouping in the Traces Dashboard.
  - metadata: untyped
    
    The arbitrary metadata to attach to this trace to enable filtering in the Traces Dashboard.
  - workflow_name: String
    
    The name of the workflow to attach to this trace. This is used to name the trace in the Traces Dashboard.
truncation: RealtimeTruncation

When the number of tokens in a conversation exceeds the model's input token limit, the conversation be truncated, meaning messages (starting from the oldest) will not be included in the model's context. A 32k context model with 4,096 max output tokens can only include 28,224 tokens in the context before truncation occurs.

Clients can configure truncation behavior to truncate with a lower max token limit, which is an effective way to control token usage and cost.

Truncation will reduce the number of cached tokens on the next turn (busting the cache), since messages are dropped from the beginning of the context. However, clients can also configure truncation to retain messages up to a fraction of the maximum context size, which will reduce the need for future truncations and thus improve the cache rate.

Truncation can be disabled entirely, which means the server will never truncate but would instead return an error if the conversation exceeds the model's input token limit.
- RealtimeTruncationStrategy = :auto | :disabled
  
  The truncation strategy to use for the session. auto is the default truncation strategy. disabled will disable truncation and emit errors when the conversation exceeds the input token limit.
  - :auto
  - :disabled
- class RealtimeTruncationRetentionRatio
  
  Retain a fraction of the conversation tokens when the conversation exceeds the input token limit. This allows you to amortize truncations across multiple turns, which can help improve cached token usage.
  - retention_ratio: Float
    
    Fraction of post-instruction conversation tokens to retain (0.0 - 1.0) when the conversation exceeds the input token limit. Setting this to 0.8 means that messages will be dropped until 80% of the maximum allowed tokens are used. This helps reduce the frequency of truncations and improve cache rates.
  - type: :retention_ratio
    
    Use retention ratio truncation.
    - :retention_ratio
  - token_limits: TokenLimits{ post_instructions}
    
    Optional custom token limits for this truncation strategy. If not provided, the model's default token limits will be used.
    - post_instructions: Integer
      
      Maximum tokens allowed in the conversation after instructions (which including tool definitions). For example, setting this to 5,000 would mean that truncation would occur when the conversation exceeds 5,000 tokens after instructions. This cannot be higher than the model's context window size minus the maximum output tokens.

Example

require "openai"

openai = OpenAI::Client.new(api_key: "My API Key")

result = openai.realtime.calls.accept("call_id", type: :realtime)

puts(result)

Hang up call

realtime.calls.hangup(call_id) -> void

post /realtime/calls/{call_id}/hangup

End an active Realtime API call, whether it was initiated over SIP or WebRTC.

Parameters

call_id: String

Example

require "openai"

openai = OpenAI::Client.new(api_key: "My API Key")

result = openai.realtime.calls.hangup("call_id")

puts(result)

Refer call

realtime.calls.refer(call_id, **kwargs) -> void

post /realtime/calls/{call_id}/refer

Transfer an active SIP call to a new destination using the SIP REFER verb.

Parameters

call_id: String
target_uri: String

URI that should appear in the SIP Refer-To header. Supports values like tel:+14155550123 or sip:agent@example.com.

Example

require "openai"

openai = OpenAI::Client.new(api_key: "My API Key")

result = openai.realtime.calls.refer("call_id", target_uri: "tel:+14155550123")

puts(result)

Reject call

realtime.calls.reject(call_id, **kwargs) -> void

post /realtime/calls/{call_id}/reject

Decline an incoming SIP call by returning a SIP status code to the caller.

Parameters

call_id: String
status_code: Integer

SIP response code to send back to the caller. Defaults to 603 (Decline) when omitted.

Example

require "openai"

openai = OpenAI::Client.new(api_key: "My API Key")

result = openai.realtime.calls.reject("call_id")

puts(result)

Translations

Client Secrets

Sessions

Transcription Sessions

ruby/resources/realtime/index.md +157 −72

5057 5057

5058 Identifier for service connectors, like those available in ChatGPT. One of5058 Identifier for service connectors, like those available in ChatGPT. One of

5059 `server_url` or `connector_id` must be provided. Learn more about service5059 `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more

5060 connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).5060 about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).

5061 5061

5062 Currently supported `connector_id` values are:5062 Currently supported `connector_id` values are:

5063 5063

5149 5149

5150 - `server_url: String`5150 - `server_url: String`

5151 5151

5152 The URL for the MCP server. One of `server_url` or `connector_id` must be5152 The URL for the MCP server. One of `server_url`, `connector_id`, or

5153 provided.5153 `tunnel_id` must be provided.

5154

5155 - `tunnel_id: String`

5156

5157 The Secure MCP Tunnel ID to use instead of a direct server URL. One of

5158 `server_url`, `connector_id`, or `tunnel_id` must be provided.

5154 5159

5155 - `class SessionUpdateEvent`5160 - `class SessionUpdateEvent`

5156 5161

5582 5587

5583 Identifier for service connectors, like those available in ChatGPT. One of5588 Identifier for service connectors, like those available in ChatGPT. One of

5584 `server_url` or `connector_id` must be provided. Learn more about service5589 `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more

5585 connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).5590 about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).

5586 5591

5587 Currently supported `connector_id` values are:5592 Currently supported `connector_id` values are:

5588 5593

5674 5679

5675 - `server_url: String`5680 - `server_url: String`

5676 5681

5677 The URL for the MCP server. One of `server_url` or `connector_id` must be5682 The URL for the MCP server. One of `server_url`, `connector_id`, or

5678 provided.5683 `tunnel_id` must be provided.

5684

5685 - `tunnel_id: String`

5686

5687 The Secure MCP Tunnel ID to use instead of a direct server URL. One of

5688 `server_url`, `connector_id`, or `tunnel_id` must be provided.

5679 5689

5680 - `tracing: RealtimeTracingConfig`5690 - `tracing: RealtimeTracingConfig`

5681 5691

7333 7343

7334 Identifier for service connectors, like those available in ChatGPT. One of7344 Identifier for service connectors, like those available in ChatGPT. One of

7335 `server_url` or `connector_id` must be provided. Learn more about service7345 `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more

7336 connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).7346 about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).

7337 7347

7338 Currently supported `connector_id` values are:7348 Currently supported `connector_id` values are:

7339 7349

7425 7435

7426 - `server_url: String`7436 - `server_url: String`

7427 7437

7428 The URL for the MCP server. One of `server_url` or `connector_id` must be7438 The URL for the MCP server. One of `server_url`, `connector_id`, or

7429 provided.7439 `tunnel_id` must be provided.

7440

7441 - `tunnel_id: String`

7442

7443 The Secure MCP Tunnel ID to use instead of a direct server URL. One of

7444 `server_url`, `connector_id`, or `tunnel_id` must be provided.

7430 7445

7431### Realtime Response Create Params7446### Realtime Response Create Params

7432 7447

8260 8275

8261 Identifier for service connectors, like those available in ChatGPT. One of8276 Identifier for service connectors, like those available in ChatGPT. One of

8262 `server_url` or `connector_id` must be provided. Learn more about service8277 `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more

8263 connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).8278 about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).

8264 8279

8265 Currently supported `connector_id` values are:8280 Currently supported `connector_id` values are:

8266 8281

8352 8367

8353 - `server_url: String`8368 - `server_url: String`

8354 8369

8355 The URL for the MCP server. One of `server_url` or `connector_id` must be8370 The URL for the MCP server. One of `server_url`, `connector_id`, or

8356 provided.8371 `tunnel_id` must be provided.

8372

8373 - `tunnel_id: String`

8374

8375 The Secure MCP Tunnel ID to use instead of a direct server URL. One of

8376 `server_url`, `connector_id`, or `tunnel_id` must be provided.

8357 8377

8358### Realtime Response Status8378### Realtime Response Status

8359 8379

10848 10868

10849 Identifier for service connectors, like those available in ChatGPT. One of10869 Identifier for service connectors, like those available in ChatGPT. One of

10850 `server_url` or `connector_id` must be provided. Learn more about service10870 `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more

10851 connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).10871 about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).

10852 10872

10853 Currently supported `connector_id` values are:10873 Currently supported `connector_id` values are:

10854 10874

10940 10960

10941 - `server_url: String`10961 - `server_url: String`

10942 10962

10943 The URL for the MCP server. One of `server_url` or `connector_id` must be10963 The URL for the MCP server. One of `server_url`, `connector_id`, or

10944 provided.10964 `tunnel_id` must be provided.

10965

10966 - `tunnel_id: String`

10967

10968 The Secure MCP Tunnel ID to use instead of a direct server URL. One of

10969 `server_url`, `connector_id`, or `tunnel_id` must be provided.

10945 10970

10946 - `tracing: RealtimeTracingConfig`10971 - `tracing: RealtimeTracingConfig`

10947 10972

12682 12707

12683 Identifier for service connectors, like those available in ChatGPT. One of12708 Identifier for service connectors, like those available in ChatGPT. One of

12684 `server_url` or `connector_id` must be provided. Learn more about service12709 `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more

12685 connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).12710 about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).

12686 12711

12687 Currently supported `connector_id` values are:12712 Currently supported `connector_id` values are:

12688 12713

12774 12799

12775 - `server_url: String`12800 - `server_url: String`

12776 12801

12777 The URL for the MCP server. One of `server_url` or `connector_id` must be12802 The URL for the MCP server. One of `server_url`, `connector_id`, or

12778 provided.12803 `tunnel_id` must be provided.

12804

12805 - `tunnel_id: String`

12806

12807 The Secure MCP Tunnel ID to use instead of a direct server URL. One of

12808 `server_url`, `connector_id`, or `tunnel_id` must be provided.

12779 12809

12780 - `tracing: RealtimeTracingConfig`12810 - `tracing: RealtimeTracingConfig`

12781 12811

12981 13011

12982 Identifier for service connectors, like those available in ChatGPT. One of13012 Identifier for service connectors, like those available in ChatGPT. One of

12983 `server_url` or `connector_id` must be provided. Learn more about service13013 `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more

12984 connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).13014 about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).

12985 13015

12986 Currently supported `connector_id` values are:13016 Currently supported `connector_id` values are:

12987 13017

13073 13103

13074 - `server_url: String`13104 - `server_url: String`

13075 13105

13076 The URL for the MCP server. One of `server_url` or `connector_id` must be13106 The URL for the MCP server. One of `server_url`, `connector_id`, or

13077 provided.13107 `tunnel_id` must be provided.

13108

13109 - `tunnel_id: String`

13110

13111 The Secure MCP Tunnel ID to use instead of a direct server URL. One of

13112 `server_url`, `connector_id`, or `tunnel_id` must be provided.

13078 13113

13079### Realtime Tools Config Union13114### Realtime Tools Config Union

13080 13115

13081- `RealtimeToolsConfigUnion = RealtimeFunctionTool | Mcp{ server_label, type, allowed_tools, 7 more}`13116- `RealtimeToolsConfigUnion = RealtimeFunctionTool | Mcp{ server_label, type, allowed_tools, 8 more}`

13082 13117

13083 Give the model access to additional tools via remote Model Context Protocol13118 Give the model access to additional tools via remote Model Context Protocol

13084 (MCP) servers. [Learn more about MCP](https://platform.openai.com/docs/guides/tools-remote-mcp).13119 (MCP) servers. [Learn more about MCP](https://platform.openai.com/docs/guides/tools-remote-mcp).

13152 13187

13153 Identifier for service connectors, like those available in ChatGPT. One of13188 Identifier for service connectors, like those available in ChatGPT. One of

13154 `server_url` or `connector_id` must be provided. Learn more about service13189 `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more

13155 connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).13190 about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).

13156 13191

13157 Currently supported `connector_id` values are:13192 Currently supported `connector_id` values are:

13158 13193

13244 13279

13245 - `server_url: String`13280 - `server_url: String`

13246 13281

13247 The URL for the MCP server. One of `server_url` or `connector_id` must be13282 The URL for the MCP server. One of `server_url`, `connector_id`, or

13248 provided.13283 `tunnel_id` must be provided.

13284

13285 - `tunnel_id: String`

13286

13287 The Secure MCP Tunnel ID to use instead of a direct server URL. One of

13288 `server_url`, `connector_id`, or `tunnel_id` must be provided.

13249 13289

13250### Realtime Tracing Config13290### Realtime Tracing Config

13251 13291

16270 16310

16271 Identifier for service connectors, like those available in ChatGPT. One of16311 Identifier for service connectors, like those available in ChatGPT. One of

16272 `server_url` or `connector_id` must be provided. Learn more about service16312 `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more

16273 connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).16313 about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).

16274 16314

16275 Currently supported `connector_id` values are:16315 Currently supported `connector_id` values are:

16276 16316

16362 16402

16363 - `server_url: String`16403 - `server_url: String`

16364 16404

16365 The URL for the MCP server. One of `server_url` or `connector_id` must be16405 The URL for the MCP server. One of `server_url`, `connector_id`, or

16366 provided.16406 `tunnel_id` must be provided.

16407

16408 - `tunnel_id: String`

16409

16410 The Secure MCP Tunnel ID to use instead of a direct server URL. One of

16411 `server_url`, `connector_id`, or `tunnel_id` must be provided.

16367 16412

16368### Response Created Event16413### Response Created Event

16369 16414

19603 19648

19604 Identifier for service connectors, like those available in ChatGPT. One of19649 Identifier for service connectors, like those available in ChatGPT. One of

19605 `server_url` or `connector_id` must be provided. Learn more about service19650 `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more

19606 connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).19651 about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).

19607 19652

19608 Currently supported `connector_id` values are:19653 Currently supported `connector_id` values are:

19609 19654

19695 19740

19696 - `server_url: String`19741 - `server_url: String`

19697 19742

19698 The URL for the MCP server. One of `server_url` or `connector_id` must be19743 The URL for the MCP server. One of `server_url`, `connector_id`, or

19699 provided.19744 `tunnel_id` must be provided.

19745

19746 - `tunnel_id: String`

19747

19748 The Secure MCP Tunnel ID to use instead of a direct server URL. One of

19749 `server_url`, `connector_id`, or `tunnel_id` must be provided.

19700 19750

19701 - `tracing: RealtimeTracingConfig`19751 - `tracing: RealtimeTracingConfig`

19702 19752

20542 20592

20543 Identifier for service connectors, like those available in ChatGPT. One of20593 Identifier for service connectors, like those available in ChatGPT. One of

20544 `server_url` or `connector_id` must be provided. Learn more about service20594 `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more

20545 connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).20595 about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).

20546 20596

20547 Currently supported `connector_id` values are:20597 Currently supported `connector_id` values are:

20548 20598

20634 20684

20635 - `server_url: String`20685 - `server_url: String`

20636 20686

20637 The URL for the MCP server. One of `server_url` or `connector_id` must be20687 The URL for the MCP server. One of `server_url`, `connector_id`, or

20638 provided.20688 `tunnel_id` must be provided.

20689

20690 - `tunnel_id: String`

20691

20692 The Secure MCP Tunnel ID to use instead of a direct server URL. One of

20693 `server_url`, `connector_id`, or `tunnel_id` must be provided.

20639 20694

20640 - `tracing: RealtimeTracingConfig`20695 - `tracing: RealtimeTracingConfig`

20641 20696

21481 21536

21482 Identifier for service connectors, like those available in ChatGPT. One of21537 Identifier for service connectors, like those available in ChatGPT. One of

21483 `server_url` or `connector_id` must be provided. Learn more about service21538 `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more

21484 connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).21539 about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).

21485 21540

21486 Currently supported `connector_id` values are:21541 Currently supported `connector_id` values are:

21487 21542

21573 21628

21574 - `server_url: String`21629 - `server_url: String`

21575 21630

21576 The URL for the MCP server. One of `server_url` or `connector_id` must be21631 The URL for the MCP server. One of `server_url`, `connector_id`, or

21577 provided.21632 `tunnel_id` must be provided.

21633

21634 - `tunnel_id: String`

21635

21636 The Secure MCP Tunnel ID to use instead of a direct server URL. One of

21637 `server_url`, `connector_id`, or `tunnel_id` must be provided.

21578 21638

21579 - `tracing: RealtimeTracingConfig`21639 - `tracing: RealtimeTracingConfig`

21580 21640

22718 22778

22719 Identifier for service connectors, like those available in ChatGPT. One of22779 Identifier for service connectors, like those available in ChatGPT. One of

22720 `server_url` or `connector_id` must be provided. Learn more about service22780 `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more

22721 connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).22781 about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).

22722 22782

22723 Currently supported `connector_id` values are:22783 Currently supported `connector_id` values are:

22724 22784

22810 22870

22811 - `server_url: String`22871 - `server_url: String`

22812 22872

22813 The URL for the MCP server. One of `server_url` or `connector_id` must be22873 The URL for the MCP server. One of `server_url`, `connector_id`, or

22814 provided.22874 `tunnel_id` must be provided.

22875

22876 - `tunnel_id: String`

22877

22878 The Secure MCP Tunnel ID to use instead of a direct server URL. One of

22879 `server_url`, `connector_id`, or `tunnel_id` must be provided.

22815 22880

22816 - `tracing: RealtimeTracingConfig`22881 - `tracing: RealtimeTracingConfig`

22817 22882

23577 23642

23578 The name of the tool to call on the server.23643 The name of the tool to call on the server.

23579 23644

23580 - `tools: Array[RealtimeFunctionTool | McpTool{ server_label, type, allowed_tools, 7 more}]`23645 - `tools: Array[RealtimeFunctionTool | McpTool{ server_label, type, allowed_tools, 8 more}]`

23581 23646

23582 Tools available to the model.23647 Tools available to the model.

23583 23648

23650 23715

23651 Identifier for service connectors, like those available in ChatGPT. One of23716 Identifier for service connectors, like those available in ChatGPT. One of

23652 `server_url` or `connector_id` must be provided. Learn more about service23717 `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more

23653 connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).23718 about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).

23654 23719

23655 Currently supported `connector_id` values are:23720 Currently supported `connector_id` values are:

23656 23721

23742 23807

23743 - `server_url: String`23808 - `server_url: String`

23744 23809

23745 The URL for the MCP server. One of `server_url` or `connector_id` must be23810 The URL for the MCP server. One of `server_url`, `connector_id`, or

23746 provided.23811 `tunnel_id` must be provided.

23812

23813 - `tunnel_id: String`

23814

23815 The Secure MCP Tunnel ID to use instead of a direct server URL. One of

23816 `server_url`, `connector_id`, or `tunnel_id` must be provided.

23747 23817

23748 - `tracing: :auto | TracingConfiguration{ group_id, metadata, workflow_name}`23818 - `tracing: :auto | TracingConfiguration{ group_id, metadata, workflow_name}`

23749 23819

24533 24603

24534 The name of the tool to call on the server.24604 The name of the tool to call on the server.

24535 24605

24536 - `tools: Array[RealtimeFunctionTool | McpTool{ server_label, type, allowed_tools, 7 more}]`24606 - `tools: Array[RealtimeFunctionTool | McpTool{ server_label, type, allowed_tools, 8 more}]`

24537 24607

24538 Tools available to the model.24608 Tools available to the model.

24539 24609

24606 24676

24607 Identifier for service connectors, like those available in ChatGPT. One of24677 Identifier for service connectors, like those available in ChatGPT. One of

24608 `server_url` or `connector_id` must be provided. Learn more about service24678 `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more

24609 connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).24679 about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).

24610 24680

24611 Currently supported `connector_id` values are:24681 Currently supported `connector_id` values are:

24612 24682

24698 24768

24699 - `server_url: String`24769 - `server_url: String`

24700 24770

24701 The URL for the MCP server. One of `server_url` or `connector_id` must be24771 The URL for the MCP server. One of `server_url`, `connector_id`, or

24702 provided.24772 `tunnel_id` must be provided.

24773

24774 - `tunnel_id: String`

24775

24776 The Secure MCP Tunnel ID to use instead of a direct server URL. One of

24777 `server_url`, `connector_id`, or `tunnel_id` must be provided.

24703 24778

24704 - `tracing: :auto | TracingConfiguration{ group_id, metadata, workflow_name}`24779 - `tracing: :auto | TracingConfiguration{ group_id, metadata, workflow_name}`

24705 24780

25527 25602

25528 The name of the tool to call on the server.25603 The name of the tool to call on the server.

25529 25604

25530 - `tools: Array[RealtimeFunctionTool | McpTool{ server_label, type, allowed_tools, 7 more}]`25605 - `tools: Array[RealtimeFunctionTool | McpTool{ server_label, type, allowed_tools, 8 more}]`

25531 25606

25532 Tools available to the model.25607 Tools available to the model.

25533 25608

25600 25675

25601 Identifier for service connectors, like those available in ChatGPT. One of25676 Identifier for service connectors, like those available in ChatGPT. One of

25602 `server_url` or `connector_id` must be provided. Learn more about service25677 `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more

25603 connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).25678 about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).

25604 25679

25605 Currently supported `connector_id` values are:25680 Currently supported `connector_id` values are:

25606 25681

25692 25767

25693 - `server_url: String`25768 - `server_url: String`

25694 25769

25695 The URL for the MCP server. One of `server_url` or `connector_id` must be25770 The URL for the MCP server. One of `server_url`, `connector_id`, or

25696 provided.25771 `tunnel_id` must be provided.

25772

25773 - `tunnel_id: String`

25774

25775 The Secure MCP Tunnel ID to use instead of a direct server URL. One of

25776 `server_url`, `connector_id`, or `tunnel_id` must be provided.

25697 25777

25698 - `tracing: :auto | TracingConfiguration{ group_id, metadata, workflow_name}`25778 - `tracing: :auto | TracingConfiguration{ group_id, metadata, workflow_name}`

25699 25779

26468 26548

26469 Identifier for service connectors, like those available in ChatGPT. One of26549 Identifier for service connectors, like those available in ChatGPT. One of

26470 `server_url` or `connector_id` must be provided. Learn more about service26550 `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more

26471 connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).26551 about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors).

26472 26552

26473 Currently supported `connector_id` values are:26553 Currently supported `connector_id` values are:

26474 26554

26560 26640

26561 - `server_url: String`26641 - `server_url: String`

26562 26642

26563 The URL for the MCP server. One of `server_url` or `connector_id` must be26643 The URL for the MCP server. One of `server_url`, `connector_id`, or

26564 provided.26644 `tunnel_id` must be provided.

26645

26646 - `tunnel_id: String`

26647

26648 The Secure MCP Tunnel ID to use instead of a direct server URL. One of

26649 `server_url`, `connector_id`, or `tunnel_id` must be provided.

26565 26650

26566- `tracing: RealtimeTracingConfig`26651- `tracing: RealtimeTracingConfig`

26567 26652

ruby/resources/realtime/index.md 2026-06-16 21:57 UTC to 2026-06-17 18:02 UTC

Realtime

Domain Types

Audio Transcription

Conversation Created Event

Conversation Item

Conversation Item Added

Conversation Item Create Event

Conversation Item Created Event

Conversation Item Delete Event

Conversation Item Deleted Event

Conversation Item Done

Conversation Item Input Audio Transcription Completed Event

Conversation Item Input Audio Transcription Delta Event

Conversation Item Input Audio Transcription Failed Event

Conversation Item Input Audio Transcription Segment

Conversation Item Retrieve Event

Conversation Item Truncate Event

Conversation Item Truncated Event

Conversation Item With Reference

Input Audio Buffer Append Event

Input Audio Buffer Clear Event

Input Audio Buffer Cleared Event

Input Audio Buffer Commit Event

Input Audio Buffer Committed Event

Input Audio Buffer Dtmf Event Received Event

Input Audio Buffer Speech Started Event

Input Audio Buffer Speech Stopped Event

Input Audio Buffer Timeout Triggered

Log Prob Properties

Mcp List Tools Completed

Mcp List Tools Failed

Mcp List Tools In Progress

Noise Reduction Type

Output Audio Buffer Clear Event

Rate Limits Updated Event

Realtime Audio Config

Realtime Audio Config Input

Realtime Audio Config Output

Realtime Audio Formats

Realtime Audio Input Turn Detection

Realtime Client Event

Realtime Conversation Item Assistant Message

Realtime Conversation Item Function Call

Realtime Conversation Item Function Call Output

Realtime Conversation Item System Message

Realtime Conversation Item User Message

Realtime Error

Realtime Error Event

Realtime Function Tool

Realtime Mcp Approval Request

Realtime Mcp Approval Response

Realtime Mcp List Tools

Realtime Mcp Protocol Error

Realtime Mcp Tool Call

Realtime Mcp Tool Execution Error

Realtime Mcphttp Error

Realtime Reasoning

Realtime Reasoning Effort

Realtime Response

Realtime Response Create Audio Output

Realtime Response Create Mcp Tool

Realtime Response Create Params

Realtime Response Status

Realtime Response Usage

Realtime Response Usage Input Token Details

Realtime Response Usage Output Token Details

Realtime Server Event

Realtime Session

Realtime Session Create Request

Realtime Tool Choice Config

Realtime Tools Config

Realtime Tools Config Union

Realtime Tracing Config

Realtime Transcription Session Audio

Realtime Transcription Session Audio Input

Realtime Transcription Session Audio Input Turn Detection

Realtime Transcription Session Create Request

Realtime Translation Client Event

Realtime Translation Client Secret Create Request