1#### Model Capabilities1#### Model Capabilities
2 2
3# Voice Agent API3# Speech to Speech API
4 4
5Build real-time voice applications powered by Grok. Stream audio and text bidirectionally via WebSocket for voice assistants, phone agents, and interactive voice systems.5Build real-time voice applications powered by Grok. Stream audio and text bidirectionally via WebSocket for voice assistants, phone agents, and interactive voice systems.
6 6
7## Quick Start7## Quick Start
8 8
9Connect to the Voice Agent API and start a conversation:9Connect to the Speech to Speech API and start a conversation:
10 10
11```python customLanguage="pythonWithoutSDK"11```python customLanguage="pythonWithoutSDK"
12import asyncio12import asyncio
149| `instructions` | string | System prompt |149| `instructions` | string | System prompt |
150| `reasoning.effort` | `"high"` | `"none"` | optional | Controls whether the model uses reasoning. Defaults to `"high"`. Supported only with `grok-voice-latest` and `grok-voice-think-fast-1.0`. |150| `reasoning.effort` | `"high"` | `"none"` | optional | Controls whether the model uses reasoning. Defaults to `"high"`. Supported only with `grok-voice-latest` and `grok-voice-think-fast-1.0`. |
151| `voice` | string | Voice selection: any built-in voice (e.g. `eve`, the default) or a [custom voice ID](/developers/model-capabilities/audio/custom-voices) (see [Available Voices](#available-voices)) |151| `voice` | string | Voice selection: any built-in voice (e.g. `eve`, the default) or a [custom voice ID](/developers/model-capabilities/audio/custom-voices) (see [Available Voices](#available-voices)) |
152| `tools` | array | Tools available to the voice agent. Supports `file_search`, `web_search`, `x_search`, `mcp`, and `function` types. See [Using Tools](#using-tools-with-grok-voice-agent-api). |152| `tools` | array | Tools available to the voice agent. Supports `file_search`, `web_search`, `x_search`, `mcp`, and `function` types. See [Using Tools](#using-tools-with-grok-speech-to-speech-api). |
153| `turn_detection.type` | string | null | `"server_vad"` for automatic detection, `null` for manual text turns |153| `turn_detection.type` | string | null | `"server_vad"` for automatic detection, `null` for manual text turns |
154| `turn_detection.threshold` | number | optional | VAD activation threshold (0.1–0.9). Higher values require louder audio to trigger. Default: `0.85`. |154| `turn_detection.threshold` | number | optional | VAD activation threshold (0.1–0.9). Higher values require louder audio to trigger. Default: `0.85`. |
155| `turn_detection.silence_duration_ms` | number | optional | How long the user must be silent (in ms) before the server ends the turn (0–10000). Higher values let users pause longer without being cut off. |155| `turn_detection.silence_duration_ms` | number | optional | How long the user must be silent (in ms) before the server ends the turn (0–10000). Higher values let users pause longer without being cut off. |
169 169
170## Available Voices170## Available Voices
171 171
172The same roster of voices works across the Voice Agent API and Text to Speech API. Browse the full list with tone descriptions and samples in the [voice table](/developers/model-capabilities/audio/text-to-speech#voices), or fetch it programmatically via [`GET /v1/tts/voices`](/developers/rest-api-reference/inference/voice). Pass the lowercase voice ID as the `voice` parameter on `session.update`.172The same roster of voices works across the Speech to Speech API and Text to Speech API. Browse the full list with tone descriptions and samples in the [voice table](/developers/model-capabilities/audio/text-to-speech#voices), or fetch it programmatically via [`GET /v1/tts/voices`](/developers/rest-api-reference/inference/voice). Pass the lowercase voice ID as the `voice` parameter on `session.update`.
173 173
174### Custom Voices174### Custom Voices
175 175
535 535
536## Supported Languages536## Supported Languages
537 537
538The Voice Agent API supports 20+ languages with native-quality accents. The model automatically detects the input language and responds naturally in the same language — no configuration required.538The Speech to Speech API supports 20+ languages with native-quality accents. The model automatically detects the input language and responds naturally in the same language — no configuration required.
539 539
540| Language | Code |540| Language | Code |
541|----------|------|541|----------|------|
632}));632}));
633```633```
634 634
635## Using Tools with Grok Voice Agent API635## Using Tools with Grok Speech to Speech API
636 636
637The Grok Voice Agent API supports various tools that can be configured in your session to enhance the capabilities of your voice agent. Tools can be configured in the `session.update` message.637The Grok Speech to Speech API supports various tools that can be configured in your session to enhance the capabilities of your voice agent. Tools can be configured in the `session.update` message.
638 638
639### Available Tool Types639### Available Tool Types
640 640
1373 1373
1374## Best Practices1374## Best Practices
1375 1375
1376This section outlines key recommendations for building low-latency, reliable, and natural-feeling voice experiences using the xAI Voice Agent API.1376This section outlines key recommendations for building low-latency, reliable, and natural-feeling voice experiences using the xAI Speech to Speech API.
1377 1377
1378### Minimize perceived latency with parallel initialization1378### Minimize perceived latency with parallel initialization
1379 1379
1494 1494
1495## SIP phone calls1495## SIP phone calls
1496 1496
1497Route PSTN, contact-center, or PBX calls into a Voice Agent API session. See [SIP Phone Calls](/developers/model-capabilities/audio/voice-agent/sip) for API integration with `CreatePhoneNumberV2`, call control, DTMF, and telephony provider examples.1497Route PSTN, contact-center, or PBX calls into a Speech to Speech API session. See [SIP Phone Calls](/developers/model-capabilities/audio/speech-to-speech/sip) for API integration with `CreatePhoneNumberV2`, call control, DTMF, and telephony provider examples.
1498 1498
1499## Migrating from OpenAI Realtime1499## Migrating from OpenAI Realtime
1500 1500
1501If you have an existing application built on the [OpenAI Realtime API](https://developers.openai.com/api/docs/guides/realtime-conversations), switching to the Grok Voice Agent API requires only a few changes: update the base URL, swap your API key, and choose a Grok voice model.1501If you have an existing application built on the [OpenAI Realtime API](https://developers.openai.com/api/docs/guides/realtime-conversations), switching to the Grok Speech to Speech API requires only a few changes: update the base URL, swap your API key, and choose a Grok voice model.
1502 1502
1503### Step 1 — Update the Base URL and API Key1503### Step 1 — Update the Base URL and API Key
1504 1504
1645 1645
1646## OpenAI Realtime API Compatibility1646## OpenAI Realtime API Compatibility
1647 1647
1648The Grok Voice Agent API is compatible with the [OpenAI Realtime API](https://developers.openai.com/api/docs/guides/realtime-conversations). Most OpenAI client libraries and SDKs work with the xAI endpoint by changing the base URL to `wss://api.x.ai/v1/realtime`. This section documents event naming differences and unsupported events.1648The Grok Speech to Speech API is compatible with the [OpenAI Realtime API](https://developers.openai.com/api/docs/guides/realtime-conversations). Most OpenAI client libraries and SDKs work with the xAI endpoint by changing the base URL to `wss://api.x.ai/v1/realtime`. This section documents event naming differences and unsupported events.
1649 1649
1650### Event Naming Differences1650### Event Naming Differences
1651 1651