SpyBara
Go Premium

Documentation 2026-10-29 23:59 UTC to 2026-10-30 23:59 UTC

4 files changed +312 −32. View all changes and history on the product overview
2026
Sat 31 23:01 Fri 30 23:59 Thu 29 23:59 Wed 28 23:00 Tue 27 23:58 Mon 26 23:58 Sat 24 23:02 Fri 23 23:58 Thu 22 22:59 Wed 21 23:02 Tue 20 23:02 Mon 19 23:00 Sun 18 23:00 Sat 17 22:57 Fri 16 23:02 Thu 15 23:00 Wed 14 23:59 Tue 13 23:00 Sun 11 23:59 Sat 10 23:58 Fri 9 22:57 Thu 8 23:01 Wed 7 23:57 Tue 6 22:58 Mon 5 22:59 Sun 4 22:58 Fri 2 23:58 Thu 1 22:59
Details

264 264 

265By default, the API supports the following file formats:265By default, the API supports the following file formats:

266 266 

267- **Images:** PNG, JPEG, and WebP.267- **Images:** PNG, JPEG, WebP, and GIF.

268- **Audio:** MP3, Opus, AAC, FLAC, WAV, and PCM.268- **Audio:** MP3, Opus, AAC, FLAC, WAV, and PCM.

269 269 

270Limit each uploaded file to 50 MiB. Audio must be 60 seconds or shorter after270Limit each uploaded file to 50 MiB. Audio must be 60 seconds or shorter after

271decoding.271decoding.

272 272 

273Set the uploaded `file` part's media type. For example, use `image/png` for a PNG273Set the uploaded `file` part's media type. Use `image/gif` for a GIF, `image/png`

274image or `audio/ogg` for Opus audio. Don't add a separate `type` field or274for a PNG image, or `audio/ogg` for Opus audio. Don't add a separate `type` field or

275manually set the `multipart/form-data` request header. The `curl` `-F` option275manually set the `multipart/form-data` request header. The `curl` `-F` option

276sets the request content type and multipart boundary. Send one file per request.276sets the request content type and multipart boundary. Send one file per request.

277 277 

Details

1372Listen for out-of-band model response with custom context1372Listen for out-of-band model response with custom context

1373 1373 

1374```javascript1374```javascript

1375const existingItemId = "item_pizza";

1376ws.send(

1377 JSON.stringify({

1378 type: "conversation.item.create",

1379 item: {

1380 id: existingItemId,

1381 type: "message",

1382 role: "user",

1383 content: [

1384 { type: "input_text", text: "We are ordering pizza for dinner." },

1385 ],

1386 },

1387 })

1388);

1375const event = {1389const event = {

1376 type: "response.create",1390 type: "response.create",

1377 response: {1391 response: {


1406```1420```

1407 1421 

1408```python1422```python

1423import json

1424 

1425existing_item_id = "item_pizza"

1426ws.send(

1427 json.dumps(

1428 {

1429 "type": "conversation.item.create",

1430 "item": {

1431 "id": existing_item_id,

1432 "type": "message",

1433 "role": "user",

1434 "content": [

1435 {"type": "input_text", "text": "We are ordering pizza for dinner."}

1436 ],

1437 },

1438 }

1439 )

1440)

1441 

1409event = {1442event = {

1410 "type": "response.create",1443 "type": "response.create",

1411 "response": {1444 "response": {


1416 # context is appropriate1449 # context is appropriate

1417 "input": [1450 "input": [

1418 # potentially include existing conversation items:1451 # potentially include existing conversation items:

1419 {"type": "item_reference", "id": "some_conversation_item_id"},1452 {"type": "item_reference", "id": existing_item_id},

1420 # include new content as well1453 # include new content as well

1421 {1454 {

1422 "type": "message",1455 "type": "message",


1436```1469```

1437 1470 

1438```ruby1471```ruby

1472existing_item_id = "item_order"

1473connection.conversation.items.create(

1474 id: existing_item_id,

1475 type: :message,

1476 role: :user,

1477 content: [

1478 {

1479 type: :input_text,

1480 text: "Order ORD-123 has not arrived."

1481 }

1482 ]

1483)

1484 

1439connection.response.create(1485connection.response.create(

1440 conversation: :none,1486 conversation: :none,

1441 metadata: { topic: "classification" },1487 metadata: { topic: "classification" },


1764 1810 

1765With a WebSocket connection the client manages audio playback, and thus must stop playback and handle truncation. Here's how this procedure works:1811With a WebSocket connection the client manages audio playback, and thus must stop playback and handle truncation. Here's how this procedure works:

1766 1812 

17671. The client monitors for new `input_audio_buffer.speech_started` events from the server, which indicate the user has started speaking. The server will automatically cancel any in-progress model response and a `response.cancelled` event will be emitted.18131. On `input_audio_buffer.speech_started`, immediately stop playback of the model's audio and discard any queued audio for that item. Record how many milliseconds of that item's audio the user actually heard.

17681. When the client detects this event, it should immediately stop playback of any audio currently being played from the model. It should note how much of the last audio response was played before the interruption.18141. Send a [`conversation.item.truncate`](https://developers.openai.com/api/reference/resources/realtime) event with the interrupted item's ID, the audio content index, and that played duration as `audio_end_ms`. This removes the unplayed portion from the conversation.

17691. The client should send a [`conversation.item.truncate`](https://developers.openai.com/api/reference/resources/realtime) event to remove the unplayed portion of the model's last response from the conversation.18151. When VAD interrupts an in-progress response, the server cancels it and sends `response.done` with `response.status` set to `cancelled`. Use that event to observe completion; don't wait for it before stopping playback or sending the truncation event.

1770 1816 

1771Here's an example:1817Here's an example:

1772 1818 

1773```json1819```json

1774{1820{

1775 "type": "conversation.item.truncate",1821 "type": "conversation.item.truncate",

1776 "item_id": "item_1234", # this is the item ID of the model's last response1822 "item_id": "item_1234",

1777 "content_index": 0,1823 "content_index": 0,

1778 "audio_end_ms": 1500 # truncate audio after 1.5 seconds1824 "audio_end_ms": 1500

1779}1825}

1780```1826```

1781 1827 

1828This example truncates audio after 1.5 seconds of playback.

1829 

1782What about truncating the transcript as well? The realtime model doesn't have enough information to precisely align transcript and audio, and thus `conversation.item.truncate` will cut the audio at a given place and remove the text transcript for the unplayed portion. This solves the problem of removing unplayed audio but doesn't provide a truncated transcript.1830What about truncating the transcript as well? The realtime model doesn't have enough information to precisely align transcript and audio, and thus `conversation.item.truncate` will cut the audio at a given place and remove the text transcript for the unplayed portion. This solves the problem of removing unplayed audio but doesn't provide a truncated transcript.

1783 1831 

1784## Push-to-talk1832## Push-to-talk

Details

123 123 

124Connect to the dedicated translation endpoint and select the model in the URL:124Connect to the dedicated translation endpoint and select the model in the URL:

125 125 

126Before running this example, install `ws` for Node.js, `websocket-client` for Python, or `async-websocket` for Ruby (`gem install async-websocket`). For Java, add the Maven dependency `com.openai:openai-java:4.75.1`.126Before running this example, install `ws` for Node.js, `websocket-client` and `sounddevice` for Python, or `async-websocket` for Ruby (`gem install async-websocket`). For Java, add the Maven dependency `com.openai:openai-java:4.75.1`.

127 127 

128The Java examples cover connection setup, language configuration, and audio input separately. They don't show a complete translation workflow: the Java SDK supports receiving events and finishing a session, but this guide doesn't include Java examples for those steps. Use the Node.js or Python examples to follow the complete workflow below.128The Java examples cover connection setup, language configuration, and audio input separately. They don't show a complete translation workflow: the Java SDK supports receiving events and finishing a session, but this guide doesn't include Java examples for those steps. Use the Node.js or Python examples to follow the complete workflow below.

129 129 


366```366```

367 367 

368```python368```python

369while True:369import base64

370import json

371from concurrent.futures import ThreadPoolExecutor

372from contextlib import suppress

373from queue import SimpleQueue

374 

375import sounddevice as sd

376 

377audio_queue = SimpleQueue()

378 

379 

380def play_audio():

381 try:

382 # Closing the stream waits for PortAudio's pending buffers to finish.

383 with sd.RawOutputStream(samplerate=24000, channels=1, dtype="int16") as stream:

384 while (pcm := audio_queue.get()) is not None:

385 stream.write(pcm)

386 except Exception:

387 # The receiver may have already closed the socket.

388 with suppress(OSError):

389 ws.abort() # Wake recv() so the receiver can report the playback failure.

390 raise

391 

392 

393with ThreadPoolExecutor(max_workers=1) as executor:

394 playback = executor.submit(play_audio)

395 try:

396 while True:

370 event = json.loads(ws.recv())397 event = json.loads(ws.recv())

398 if playback.done():

399 playback.result() # Report a playback failure on the receiver.

371 400 

372 if event["type"] == "session.output_audio.delta":401 if event["type"] == "session.output_audio.delta":

373 play_pcm16(event["delta"])402 audio_queue.put(base64.b64decode(event["delta"]))

374 403 

375 if event["type"] == "session.output_transcript.delta":404 if event["type"] in (

405 "session.output_transcript.delta",

406 "session.input_transcript.delta",

407 ):

376 print(event["delta"], end="", flush=True)408 print(event["delta"], end="", flush=True)

377 409 

378 if event["type"] == "session.input_transcript.delta":410 if event["type"] == "error":

379 update_source_transcript(event["delta"])411 raise RuntimeError(f"Translation error: {event['error']['code']}")

412 

413 if event["type"] == "session.closed":

414 break

415 finally:

416 ws.close()

417 audio_queue.put(None)

418 playback.result()

380```419```

381 420 

382 421 


451```490```

452 491 

453```python492```python

493import base64

494import json

454import time495import time

496from concurrent.futures import ThreadPoolExecutor

497from contextlib import suppress

498from queue import SimpleQueue

499 

500import sounddevice as sd

501 

502audio_queue = SimpleQueue()

503 

504 

505def play_audio():

506 try:

507 # Closing the stream waits for PortAudio's pending buffers to finish.

508 with sd.RawOutputStream(samplerate=24000, channels=1, dtype="int16") as stream:

509 while (pcm := audio_queue.get()) is not None:

510 stream.write(pcm)

511 except Exception:

512 # The receiver may have already closed the socket.

513 with suppress(OSError):

514 ws.abort() # Wake recv() so the receiver can report the playback failure.

515 raise

516 

455 517 

456translation_session_closing = False518translation_session_closing = False

457 519 


465 ws.send(json.dumps({"type": "session.close"}))527 ws.send(json.dumps({"type": "session.close"}))

466 528 

467 529 

468# Call this when the source stream ends.530with ThreadPoolExecutor(max_workers=1) as executor:

469close_translation_session()531 playback = executor.submit(play_audio)

470# Adjust this deadline for your application's finalization requirements.532 try:

471drain_deadline = time.monotonic() + 15.0533 # Call this when the source stream ends.

534 close_translation_session()

535 # Adjust this deadline for your application's finalization requirements.

536 drain_deadline = time.monotonic() + 15.0

472 537 

473try:

474 while True:538 while True:

475 remaining = drain_deadline - time.monotonic()539 remaining = drain_deadline - time.monotonic()

476 if remaining <= 0:540 if remaining <= 0:


480 if not data:544 if not data:

481 raise RuntimeError("Translation disconnected before session.closed")545 raise RuntimeError("Translation disconnected before session.closed")

482 event = json.loads(data)546 event = json.loads(data)

547 if playback.done():

548 playback.result() # Report a playback failure on the receiver.

483 549 

484 if event["type"] == "error":550 if event["type"] == "error":

485 raise RuntimeError(f"Translation error: {event['error']['code']}")551 raise RuntimeError(f"Translation error: {event['error']['code']}")

486 552 

487 if event["type"] == "session.output_audio.delta":553 if event["type"] == "session.output_audio.delta":

488 play_pcm16(event["delta"])554 audio_queue.put(base64.b64decode(event["delta"]))

489 555 

490 if event["type"] == "session.output_transcript.delta":556 if event["type"] in (

557 "session.output_transcript.delta",

558 "session.input_transcript.delta",

559 ):

491 print(event["delta"], end="", flush=True)560 print(event["delta"], end="", flush=True)

492 561 

493 if event["type"] == "session.input_transcript.delta":

494 update_source_transcript(event["delta"])

495 

496 if event["type"] == "session.closed":562 if event["type"] == "session.closed":

497 break563 break

498finally:564 finally:

565 # The server deadline covers receiving events, not local playback.

499 ws.close()566 ws.close()

567 audio_queue.put(None)

568 playback.result()

500```569```

501 570 

502 571 

Details

15 15 

16Set `model` to `gpt-6-astra` or `gpt-6.1-sol` and `service_tier` to `ultrafast` in each `response.create` event.16Set `model` to `gpt-6-astra` or `gpt-6.1-sol` and `service_tier` to `ultrafast` in each `response.create` event.

17 17 

18Install your language's SDK and WebSocket dependencies as described in [Connect and create responses](https://developers.openai.com/api/docs/guides/websocket-mode#connect-and-create-responses). Set `OPENAI_API_KEY` in your environment before running the example.

19 

18Use Ultrafast across turns on one WebSocket20Use Ultrafast across turns on one WebSocket

19 21 

20```javascript22```javascript


105 raise RuntimeError("Connection closed before the response finished.")111 raise RuntimeError("Connection closed before the response finished.")

106```112```

107 113 

114```go

115package main

116 

117import (

118 "context"

119 "fmt"

120 "time"

121 

122 "github.com/openai/openai-go/v3"

123 "github.com/openai/openai-go/v3/responses"

124)

125 

126func main() {

127 ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)

128 defer cancel()

129 

130 client := openai.NewClient()

131 conn, err := client.Responses.Connect(ctx, responses.ResponseConnectionOptions{})

132 if err != nil {

133 panic(err)

134 }

135 defer conn.Close()

136 

137 previousResponseID := ""

138 

139 for _, prompt := range []string{

140 "Explain why the sky is blue in one sentence.",

141 "Now explain why sunsets look red.",

142 } {

143 request := responses.ResponsesClientEventResponseCreateParam{

144 Model: "gpt-6-astra",

145 ServiceTier: "ultrafast",

146 Input: responses.ResponsesClientEventResponseCreateInputUnionParam{

147 OfString: openai.String(prompt),

148 },

149 }

150 if previousResponseID != "" {

151 request.PreviousResponseID = openai.String(previousResponseID)

152 }

153 

154 if err := conn.Create(ctx, request); err != nil {

155 panic(err)

156 }

157 

158 // FinalResponse reads events until the response finishes.

159 response, err := conn.FinalResponse(ctx)

160 if err != nil {

161 panic(err)

162 }

163 if response.Status != responses.ResponseStatusCompleted {

164 panic(fmt.Sprintf("Response ended with status %s", response.Status))

165 }

166 

167 previousResponseID = response.ID

168 fmt.Println(response.OutputText())

169 }

170}

171```

172 

173```java

174import com.openai.client.okhttp.OpenAIOkHttpClient;

175import com.openai.models.responses.ResponseStatus;

176import com.openai.models.responses.ResponsesClientEvent;

177import java.util.List;

178 

179var client = OpenAIOkHttpClient.fromEnv();

180 

181try (var connection = client.responses().connect()) {

182 String previousResponseId = null;

183 

184 for (var prompt :

185 List.of(

186 "Explain why the sky is blue in one sentence.",

187 "Now explain why sunsets look red.")) {

188 var request =

189 ResponsesClientEvent.ResponseCreate.builder()

190 .model("gpt-6-astra")

191 .serviceTier(ResponsesClientEvent.ResponseCreate.ServiceTier.ULTRAFAST)

192 .input(prompt);

193 if (previousResponseId != null) {

194 request.previousResponseId(previousResponseId);

195 }

108 196 

109The example streams two responses over the same connection. The second request sends the new prompt and passes the first response's ID as `previous_response_id`. Reuse the connection for later turns and tool results. See [Continue with incremental inputs](https://developers.openai.com/api/docs/guides/websocket-mode#continue-with-incremental-inputs).197 connection.send(ResponsesClientEvent.ofResponseCreate(request.build()));

198 

199 var response = connection.finalResponse();

200 if (response.status().filter(ResponseStatus.COMPLETED::equals).isEmpty()) {

201 throw new IllegalStateException(

202 "Response ended with status " + response.status().orElse(null));

203 }

204 

205 previousResponseId = response.id();

206 response.output().stream()

207 .flatMap(item -> item.message().stream())

208 .flatMap(message -> message.content().stream())

209 .flatMap(content -> content.outputText().stream())

210 .forEach(text -> System.out.println(text.text()));

211 }

212} finally {

213 client.close();

214}

215```

216 

217```ruby

218# Install: gem install openai async-websocket

219# Set OPENAI_API_KEY in your environment.

220 

221require "async"

222require "openai"

223 

224client = OpenAI::Client.new

225 

226request = {

227 model: "gpt-6-astra",

228 service_tier: :ultrafast

229}

230prompts = [

231 "Explain why the sky is blue in one sentence.",

232 "Now explain why sunsets look red."

233]

234 

235Sync do |task|

236 task.with_timeout(120) do

237 client.responses.connect(request_options: { timeout: 10 }) do |connection|

238 prompts.each do |prompt|

239 connection.response.create(**request, input: prompt)

240 

241 loop do

242 event = connection.receive

243 raise "Connection closed before the response finished" unless event

244 

245 case event

246 when OpenAI::Responses::ResponseCompletedEvent

247 request[:previous_response_id] = event.response.id

248 puts(event.response.output_text)

249 break

250 when OpenAI::Responses::ResponseFailedEvent, OpenAI::Responses::ResponseIncompleteEvent,

251 OpenAI::Responses::ResponsesServerEvent::ResponseWsError

252 raise "Response failed: #{event.to_json}"

253 end

254 end

255 end

256 end

257 end

258end

259```

260 

261 

262The examples send two requests over the same connection and print each response. The Go, Java, and Ruby examples wait for the complete response before printing it. The second request sends the new prompt and passes the first response's ID as `previous_response_id`. Reuse the connection for later turns and tool results. See [Continue with incremental inputs](https://developers.openai.com/api/docs/guides/websocket-mode#continue-with-incremental-inputs).

110 263 

111## HTTP alternative264## HTTP alternative

112 265 


142```296```

143 297 

144```go298```go

145client := openai.NewClient()299package main

146response, err := client.Responses.New(context.Background(), responses.ResponseNewParams{300 

301import (

302 "context"

303 "fmt"

304 

305 "github.com/openai/openai-go/v3"

306 "github.com/openai/openai-go/v3/responses"

307)

308 

309func main() {

310 client := openai.NewClient()

311 

312 response, err := client.Responses.New(context.Background(), responses.ResponseNewParams{

147 Model: "gpt-6-astra",313 Model: "gpt-6-astra",

148 ServiceTier: responses.ResponseNewParamsServiceTierUltrafast,314 ServiceTier: responses.ResponseNewParamsServiceTierUltrafast,

149 Input: responses.ResponseNewParamsInputUnion{OfString: openai.String("Explain why the sky is blue in one sentence.")},315 Input: responses.ResponseNewParamsInputUnion{OfString: openai.String("Explain why the sky is blue in one sentence.")},

150})316 })

151if err != nil {317 if err != nil {

152 panic(err)318 panic(err)

319 }

320 

321 fmt.Println(response.OutputText())

153}322}

154fmt.Println(response.OutputText())

155```323```

156 324 

157```java325```java