guides/realtime-server-controls.md +0 −109 deleted
File Deleted View Diff
1# Webhooks and server-side controls
2
3> For the complete documentation index, see [llms.txt](/llms.txt). Markdown versions of documentation pages are available by appending `.md` to the page URL.
4
5The Realtime API allows clients to connect directly to the API server via WebRTC or SIP. However, you'll most likely want tool use and other business logic to reside on your application server to keep this logic private and client-agnostic.
6
7Keep tool use, business logic, and other details secure on the server side by connecting over a “sideband” control channel. We now have sideband options for both SIP and WebRTC connections.
8
9A sideband connection means there are two active connections to the same Realtime session: one from the user's client and one from your application server. The server connection can be used to monitor the session, update instructions, and respond to tool calls.
10
11## With WebRTC
12
131. When [establishing a peer connection](https://developers.openai.com/api/docs/guides/realtime-webrtc) you fetch and receive an SDP response from the Realtime API to configure the connection. If you used the sample code from the WebRTC guide, that looks something like this:
14
15```javascript
16const baseUrl = "https://api.openai.com/v1/realtime/calls";
17const sdpResponse = await fetch(baseUrl, {
18 method: "POST",
19 body: offer.sdp,
20 headers: {
21 Authorization: `Bearer ${EPHEMERAL_KEY}`,
22 "Content-Type": "application/sdp",
23 },
24});
25```
26
27
282. The fetch response will contain a `Location` header that has a unique call ID that can be used on the server to establish a WebSocket connection to that same Realtime session.
29
30```javascript
31// Location: /v1/realtime/calls/rtc_123456
32const location = sdpResponse.headers.get("Location");
33const callId = location?.split("/").pop();
34console.log(callId);
35```
36
37
383. On a server, you can then [listen for events and configure the session](https://developers.openai.com/api/docs/guides/realtime-conversations) just as you would from a typical Realtime API WebSocket connection, using that call ID with the URL
39 `wss://api.openai.com/v1/realtime?call_id=rtc_xxxxx`, as shown below:
40
41```javascript
42import WebSocket from "ws";
43const callId = "rtc_u1_9c6574da8b8a41a18da9308f4ad974ce";
44
45// Connect to a WebSocket for the in-progress call
46const url = "wss://api.openai.com/v1/realtime?call_id=" + callId;
47const ws = new WebSocket(url, {
48 headers: {
49 Authorization: "Bearer " + process.env.OPENAI_API_KEY,
50 },
51});
52
53ws.on("open", function open() {
54 console.log("Connected to server.");
55
56 // Send client events over the WebSocket once connected
57 ws.send(
58 JSON.stringify({
59 type: "session.update",
60 session: {
61 type: "realtime",
62 instructions: "Be extra nice today!",
63 },
64 })
65 );
66});
67
68// Listen for and parse server events
69ws.on("message", function incoming(message) {
70 console.log(JSON.parse(message.toString()));
71});
72```
73
74
75In this way, you are able to add tools, monitor sessions, and carry out business logic on the server instead of needing to configure those actions on the client.
76
77## With SIP
78
791. A user connects to OpenAI via phone over SIP.
802. OpenAI sends a webhook to your application’s server webhook URL, notifying your app of the state of the session. The webhook will look something like:
81
82```json
83POST https://my_website.com/webhook_endpoint
84user-agent: OpenAI/1.0 (+https://platform.openai.com/docs/webhooks)
85content-type: application/json
86webhook-id: wh_685342e6c53c8190a1be43f081506c52 # unique id for idempotency
87webhook-timestamp: 1750287078 # timestamp of delivery attempt
88webhook-signature: v1,K5oZfzN95Z9UVu1EsfQmfVNQhnkZ2pj9o9NDN/H/pI4= # signature to verify authenticity from OpenAI
89
90{
91 "object": "event",
92 "id": "evt_685343a1381c819085d44c354e1b330e",
93 "type": "realtime.call.incoming",
94 "created_at": 1750287018, // Unix timestamp
95 "data": {
96 "call_id": "some_unique_id",
97 "sip_headers": [
98 { "name": "From", "value": "sip:+142555512112@sip.example.com" },
99 { "name": "To", "value": "sip:+18005551212@sip.example.com" },
100 { "name": "Call-ID", "value": "03782086-4ce9-44bf-8b0d-4e303d2cc590"}
101 ]
102 }
103}
104
105```
106
1073. The application server opens a WebSocket connection to the Realtime API using the `call_id` value provided in the webhook, via a URL like this: `wss://api.openai.com/v1/realtime?call_id={callId}`. The WebSocket connection will live for the life of the SIP call.
108
109The WebSocket connection can then be used to send and receive events to control the call, just as you would if the session was initiated with a WebSocket connection. This includes monitoring the call, updating instructions dynamically, and responding to tool calls.