Documentation 2026-10-07 23:59 UTC to 2026-10-08 23:58 UTC

16 16 

17You are unable to concurrently run your requests beyond the rate limits shown in the API console.17You are unable to concurrently run your requests beyond the rate limits shown in the API console.

18 18 

19```pythonXAI

20import asyncio

21import os

22 

23from xai_sdk import AsyncClient

24from xai_sdk.chat import Response, user

25 

26async def main():

27 client = AsyncClient(

28 api_key=os.getenv("XAI_API_KEY"),

29 timeout=3600, # Override default timeout with longer timeout for reasoning models

30 )

31 

32 model = "grok-4.7"

33 requests = [

34 "Tell me a joke",

35 "Write a funny haiku",

36 "Generate a funny X post",

37 "Say something unhinged",

38 ]

39 # Define a semaphore to limit concurrent requests (e.g., max 2 concurrent requests at a time)

40 max_in_flight_requests = 2

41 semaphore = asyncio.Semaphore(max_in_flight_requests)

42 

43 async def process_request(request) -> Response:

44 async with semaphore:

45 print(f"Processing request: {request}")

46 chat = client.chat.create(model=model, max_tokens=100)

47 chat.append(user(request))

48 return await chat.sample()

49 

50 tasks = []

51 for request in requests:

52 tasks.append(process_request(request))

53 

54 responses = await asyncio.gather(*tasks)

55 for i, response in enumerate(responses):

56 print(f"Total tokens used for response {i}: {response.usage.total_tokens}")

57 

58if __name__ == "__main__":

59 asyncio.run(main())

60```

61 

62```pythonOpenAISDK19```pythonOpenAISDK

63import asyncio20import asyncio

64import os21import os

116if __name__ == "__main__":73if __name__ == "__main__":

117 asyncio.run(main())74 asyncio.run(main())

118```75```

76 

77```pythonXAI

78import asyncio

79import os

80 

81from xai_sdk import AsyncClient

82from xai_sdk.chat import Response, user

83 

84async def main():

85 client = AsyncClient(

86 api_key=os.getenv("XAI_API_KEY"),

87 timeout=3600, # Override default timeout with longer timeout for reasoning models

88 )

89 

90 model = "grok-4.7"

91 requests = [

92 "Tell me a joke",

93 "Write a funny haiku",

94 "Generate a funny X post",

95 "Say something unhinged",

96 ]

97 # Define a semaphore to limit concurrent requests (e.g., max 2 concurrent requests at a time)

98 max_in_flight_requests = 2

99 semaphore = asyncio.Semaphore(max_in_flight_requests)

100 

101 async def process_request(request) -> Response:

102 async with semaphore:

103 print(f"Processing request: {request}")

104 chat = client.chat.create(model=model, max_tokens=100)

105 chat.append(user(request))

106 return await chat.sample()

107 

108 tasks = []

109 for request in requests:

110 tasks.append(process_request(request))

111 

112 responses = await asyncio.gather(*tasks)

113 for i, response in enumerate(responses):

114 print(f"Total tokens used for response {i}: {response.usage.total_tokens}")

115 

116if __name__ == "__main__":

117 asyncio.run(main())

118```

67 }'67 }'

68```68```

69 69 

70```pythonXAI

71from xai_sdk import Client

72 

73client = Client()

74 

75# Create a batch with a descriptive name

76batch = client.batch.create(batch_name="customer_feedback_analysis")

77print(f"Created batch: {batch.batch_id}")

78 

79# Store the batch_id for later use

80batch_id = batch.batch_id

81```

82 

83```javascriptWithoutSDK70```javascriptWithoutSDK

84// Create a batch with a descriptive name71// Create a batch with a descriptive name

85const response = await fetch("https://api.x.ai/v1/batches", {72const response = await fetch("https://api.x.ai/v1/batches", {

97const batchId = batch.batch_id;84const batchId = batch.batch_id;

98```85```

99 86 

100## Step 2: Add requests to the batch

101 

102With your batch created, you can now add requests to it. Each request will be processed asynchronously.

103 

104**With the xAI SDK, adding batch requests is simple:** use `chat.create()` for text, `image.prepare()` for images, `video.prepare()` for videos, or `video.prepare_extension()` for video extensions, then pass them as a list. You can also upload a [JSONL file](#jsonl-file-upload) if you prefer.

105 

106**Important:** Assign a unique `batch_request_id` to each request. This ID lets you match results back to their original requests, which becomes important when you're processing hundreds or thousands of items. If you don't provide an ID, we generate a UUID for you. Using your own IDs is useful for idempotency (ensuring a request is only processed once) and for linking batch requests to records in your own system.

107 

108```pythonXAI87```pythonXAI

109from xai_sdk import Client88from xai_sdk import Client

110from xai_sdk.chat import system, user

111from xai_sdk.tools import web_search, x_search, mcp

112 89 

113client = Client()90client = Client()

114 91 

115batch_requests = []92# Create a batch with a descriptive name

116 93batch = client.batch.create(batch_name="customer_feedback_analysis")

117# Chat completion with tools94print(f"Created batch: {batch.batch_id}")

118chat = client.chat.create(

119 model="grok-4.3",

120 batch_request_id="chat_001",

121 tools=[web_search(), x_search()],

122)

123chat.append(system("Analyze market sentiment from recent news and posts."))

124chat.append(user("What is the current sentiment around TSLA stock?"))

125batch_requests.append(chat)

126 

127# Image generation

128image_req = client.image.prepare(

129 prompt="A sleek modern laptop on a minimalist desk",

130 model="grok-imagine-image-2.0",

131 batch_request_id="img_001",

132)

133batch_requests.append(image_req)

134 

135# Image edit

136image_edit_req = client.image.prepare(

137 prompt="Add a rainbow in the background",

138 model="grok-imagine-image-2.0",

139 image_url="https://picsum.photos/800",

140 batch_request_id="img_edit_001",

141)

142batch_requests.append(image_edit_req)

143 95 

144# Video generation96# Store the batch_id for later use

145video_req = client.video.prepare(97batch_id = batch.batch_id

146 prompt="A product rotating on a turntable with dramatic lighting",98```

147 model="grok-imagine-video-1.5",

148 batch_request_id="vid_001",

149)

150batch_requests.append(video_req)

151 99 

152# Video edit100## Step 2: Add requests to the batch

153video_edit_req = client.video.prepare(

154 prompt="Make it slow motion",

155 model="grok-imagine-video",

156 video_url="https://lorem.video/cat_360p_3s",

157 batch_request_id="vid_edit_001",

158)

159batch_requests.append(video_edit_req)

160 101 

161# Video extension102With your batch created, you can now add requests to it. Each request will be processed asynchronously.

162video_ext_req = client.video.prepare_extension(

163 prompt="The camera slowly pans to reveal a sunset behind the mountains",

164 model="grok-imagine-video",

165 video_url="https://lorem.video/cat_360p_3s",

166 duration=6,

167 batch_request_id="vid_ext_001",

168)

169batch_requests.append(video_ext_req)

170 103 

171# Remote MCP104**With the xAI SDK, adding batch requests is simple:** use `chat.create()` for text, `image.prepare()` for images, `video.prepare()` for videos, or `video.prepare_extension()` for video extensions, then pass them as a list. You can also upload a [JSONL file](#jsonl-file-upload) if you prefer.

172mcp_chat = client.chat.create(

173 model="grok-4.3",

174 batch_request_id="mcp_001",

175 tools=[mcp(server_url="https://mcp.deepwiki.com/mcp")],

176)

177mcp_chat.append(user("What does the xai-sdk-python repo do?"))

178batch_requests.append(mcp_chat)

179 105 

180# Add all requests to the batch106**Important:** Assign a unique `batch_request_id` to each request. This ID lets you match results back to their original requests, which becomes important when you're processing hundreds or thousands of items. If you don't provide an ID, we generate a UUID for you. Using your own IDs is useful for idempotency (ensuring a request is only processed once) and for linking batch requests to records in your own system.

181client.batch.add(batch_id=batch.batch_id, batch_requests=batch_requests)

182print(f"Added {len(batch_requests)} requests to batch")

183```

184 107 

185```bash108```bash

186curl -X POST https://api.x.ai/v1/batches/{batch_id}/requests \\109curl -X POST https://api.x.ai/v1/batches/{batch_id}/requests \\

318console.log(\`Added \${batchRequests.length} requests to batch\`);241console.log(\`Added \${batchRequests.length} requests to batch\`);

319```242```

320 243 

244```pythonXAI

245from xai_sdk import Client

246from xai_sdk.chat import system, user

247from xai_sdk.tools import web_search, x_search, mcp

248 

249client = Client()

250 

251batch_requests = []

252 

253# Chat completion with tools

254chat = client.chat.create(

255 model="grok-4.3",

256 batch_request_id="chat_001",

257 tools=[web_search(), x_search()],

258)

259chat.append(system("Analyze market sentiment from recent news and posts."))

260chat.append(user("What is the current sentiment around TSLA stock?"))

261batch_requests.append(chat)

262 

263# Image generation

264image_req = client.image.prepare(

265 prompt="A sleek modern laptop on a minimalist desk",

266 model="grok-imagine-image-2.0",

267 batch_request_id="img_001",

268)

269batch_requests.append(image_req)

270 

271# Image edit

272image_edit_req = client.image.prepare(

273 prompt="Add a rainbow in the background",

274 model="grok-imagine-image-2.0",

275 image_url="https://picsum.photos/800",

276 batch_request_id="img_edit_001",

277)

278batch_requests.append(image_edit_req)

279 

280# Video generation

281video_req = client.video.prepare(

282 prompt="A product rotating on a turntable with dramatic lighting",

283 model="grok-imagine-video-1.5",

284 batch_request_id="vid_001",

285)

286batch_requests.append(video_req)

287 

288# Video edit

289video_edit_req = client.video.prepare(

290 prompt="Make it slow motion",

291 model="grok-imagine-video",

292 video_url="https://lorem.video/cat_360p_3s",

293 batch_request_id="vid_edit_001",

294)

295batch_requests.append(video_edit_req)

296 

297# Video extension

298video_ext_req = client.video.prepare_extension(

299 prompt="The camera slowly pans to reveal a sunset behind the mountains",

300 model="grok-imagine-video",

301 video_url="https://lorem.video/cat_360p_3s",

302 duration=6,

303 batch_request_id="vid_ext_001",

304)

305batch_requests.append(video_ext_req)

306 

307# Remote MCP

308mcp_chat = client.chat.create(

309 model="grok-4.3",

310 batch_request_id="mcp_001",

311 tools=[mcp(server_url="https://mcp.deepwiki.com/mcp")],

312)

313mcp_chat.append(user("What does the xai-sdk-python repo do?"))

314batch_requests.append(mcp_chat)

315 

316# Add all requests to the batch

317client.batch.add(batch_id=batch.batch_id, batch_requests=batch_requests)

318print(f"Added {len(batch_requests)} requests to batch")

319```

320 

321## Step 3: Monitor batch progress321## Step 3: Monitor batch progress

322 322 

323After adding requests, they begin processing in the background. Since batch processing is asynchronous, you need to poll the batch status to know when results are ready.323After adding requests, they begin processing in the background. Since batch processing is asynchronous, you need to poll the batch status to know when results are ready.

340# }340# }

341```341```

342 342 

343```javascriptWithoutSDK

344// Poll until all requests are processed

345console.log("Waiting for batch to complete...");

346const interval = setInterval(async () => {

347 const response = await fetch(

348 \`https://api.x.ai/v1/batches/\${batchId}\`,

349 { headers: { Authorization: \`Bearer \${process.env.XAI_API_KEY}\` } }

350 );

351 const batch = await response.json();

352 

353 const { num_pending, num_success, num_error, num_requests } = batch.state;

354 const completed = num_success + num_error;

355 console.log(\`Progress: \${completed}/\${num_requests} complete, \${num_pending} pending\`);

356 

357 if (num_requests > 0 && num_pending === 0) {

358 clearInterval(interval);

359 console.log("Batch processing complete!");

360 }

361 // Wait before polling again (avoid hammering the API)

362}, 5000);

363```

364 

343```pythonXAI365```pythonXAI

344import time366import time

345from xai_sdk import Client367from xai_sdk import Client

365 time.sleep(5)387 time.sleep(5)

366```388```

367 389 

368```javascriptWithoutSDK

369// Poll until all requests are processed

370console.log("Waiting for batch to complete...");

371const interval = setInterval(async () => {

372 const response = await fetch(

373 \`https://api.x.ai/v1/batches/\${batchId}\`,

374 { headers: { Authorization: \`Bearer \${process.env.XAI_API_KEY}\` } }

375 );

376 const batch = await response.json();

377 

378 const { num_pending, num_success, num_error, num_requests } = batch.state;

379 const completed = num_success + num_error;

380 console.log(\`Progress: \${completed}/\${num_requests} complete, \${num_pending} pending\`);

381 

382 if (num_requests > 0 && num_pending === 0) {

383 clearInterval(interval);

384 console.log("Batch processing complete!");

385 }

386 // Wait before polling again (avoid hammering the API)

387}, 5000);

388```

389 

390### Understanding batch states390### Understanding batch states

391 391 

392The Batch API tracks state at two levels: the **batch level** and the **individual request level**.392The Batch API tracks state at two levels: the **batch level** and the **individual request level**.

425 425 

426**Pagination:** Results are returned in pages. Use the `limit` parameter to control page size and `pagination_token` to fetch subsequent pages. When `pagination_token` is `None`, you've reached the end.426**Pagination:** Results are returned in pages. Use the `limit` parameter to control page size and `pagination_token` to fetch subsequent pages. When `pagination_token` is `None`, you've reached the end.

427 427 

428```pythonXAI

429from xai_sdk import Client

430 

431client = Client()

432 

433# Paginate through all results

434all_succeeded = []

435all_failed = []

436pagination_token = None

437 

438while True:

439 # Fetch a page of results (limit controls page size)

440 page = client.batch.list_batch_results(

441 batch_id=batch.batch_id,

442 limit=100,

443 pagination_token=pagination_token,

444 )

445

446 # Collect results from this page

447 all_succeeded.extend(page.succeeded)

448 all_failed.extend(page.failed)

449

450 # Check if there are more pages

451 if page.pagination_token is None:

452 break

453 pagination_token = page.pagination_token

454 

455# Process results - handle different response types

456print(f"Successfully processed: {len(all_succeeded)} requests")

457for result in all_succeeded:

458 rid = result.batch_request_id

459 resp = result.proto.response

460 

461 if resp.HasField("completion_response"):

462 # Chat completion response

463 print(f"[{rid}] {result.response.content}")

464 print(f" Tokens used: {result.response.usage.total_tokens}")

465 elif resp.HasField("image_response"):

466 # Image generation response

467 print(f"[{rid}] Image URL: {result.image_response.url}")

468 elif resp.HasField("video_response"):

469 # Video generation response

470 print(f"[{rid}] Video URL: {result.video_response.url}")

471 

472if all_failed:

473 print(f"\\nFailed: {len(all_failed)} requests")

474 for result in all_failed:

475 print(f"[{result.batch_request_id}] Error: {result.error_message}")

476```

477 

478```bash428```bash

479# Fetch first page429# Fetch first page

480curl "https://api.x.ai/v1/batches/{batch_id}/results?limit=100" \\430curl "https://api.x.ai/v1/batches/{batch_id}/results?limit=100" \\

538}488}

539```489```

540 490 

491```pythonXAI

492from xai_sdk import Client

493 

494client = Client()

495 

496# Paginate through all results

497all_succeeded = []

498all_failed = []

499pagination_token = None

500 

501while True:

502 # Fetch a page of results (limit controls page size)

503 page = client.batch.list_batch_results(

504 batch_id=batch.batch_id,

505 limit=100,

506 pagination_token=pagination_token,

507 )

508

509 # Collect results from this page

510 all_succeeded.extend(page.succeeded)

511 all_failed.extend(page.failed)

512

513 # Check if there are more pages

514 if page.pagination_token is None:

515 break

516 pagination_token = page.pagination_token

517 

518# Process results - handle different response types

519print(f"Successfully processed: {len(all_succeeded)} requests")

520for result in all_succeeded:

521 rid = result.batch_request_id

522 resp = result.proto.response

523 

524 if resp.HasField("completion_response"):

525 # Chat completion response

526 print(f"[{rid}] {result.response.content}")

527 print(f" Tokens used: {result.response.usage.total_tokens}")

528 elif resp.HasField("image_response"):

529 # Image generation response

530 print(f"[{rid}] Image URL: {result.image_response.url}")

531 elif resp.HasField("video_response"):

532 # Video generation response

533 print(f"[{rid}] Video URL: {result.video_response.url}")

534 

535if all_failed:

536 print(f"\\nFailed: {len(all_failed)} requests")

537 for result in all_failed:

538 print(f"[{result.batch_request_id}] Error: {result.error_message}")

539```

540 

541## Additional operations541## Additional operations

542 542 

543Beyond the core workflow, the Batch API provides additional operations for managing your batches.543Beyond the core workflow, the Batch API provides additional operations for managing your batches.

551 -H "Authorization: Bearer $XAI_API_KEY"551 -H "Authorization: Bearer $XAI_API_KEY"

552```552```

553 553 

554```pythonXAI

555from xai_sdk import Client

556 

557client = Client()

558 

559# Cancel processing

560cancelled_batch = client.batch.cancel(batch_id=batch.batch_id)

561print(f"Cancelled batch: {cancelled_batch.batch_id}")

562print(f"Completed before cancellation: {cancelled_batch.state.num_success} requests")

563```

564 

565```javascriptWithoutSDK554```javascriptWithoutSDK

566// Cancel processing555// Cancel processing

567const response = await fetch(556const response = await fetch(

573console.log(\`Completed before cancellation: \${cancelledBatch.state.num_success} requests\`);562console.log(\`Completed before cancellation: \${cancelledBatch.state.num_success} requests\`);

574```563```

575 564 

565```pythonXAI

566from xai_sdk import Client

567 

568client = Client()

569 

570# Cancel processing

571cancelled_batch = client.batch.cancel(batch_id=batch.batch_id)

572print(f"Cancelled batch: {cancelled_batch.batch_id}")

573print(f"Completed before cancellation: {cancelled_batch.state.num_success} requests")

574```

575 

576### List all batches576### List all batches

577 577 

578View all batches belonging to your team. Batches are retained until they expire (check the `expires_at` field). This endpoint supports the same `limit` and `pagination_token` parameters for paginating through large lists.578View all batches belonging to your team. Batches are retained until they expire (check the `expires_at` field). This endpoint supports the same `limit` and `pagination_token` parameters for paginating through large lists.

582 -H "Authorization: Bearer $XAI_API_KEY"582 -H "Authorization: Bearer $XAI_API_KEY"

583```583```

584 584 

585```pythonXAI

586from xai_sdk import Client

587 

588client = Client()

589 

590# List recent batches

591response = client.batch.list(limit=20)

592 

593for batch in response.batches:

594 status = "complete" if batch.state.num_pending == 0 else "processing"

595 print(f"{batch.name} ({batch.batch_id}): {status}")

596```

597 

598```javascriptWithoutSDK585```javascriptWithoutSDK

599// List recent batches586// List recent batches

600const response = await fetch(587const response = await fetch(

609}596}

610```597```

611 598 

599```pythonXAI

600from xai_sdk import Client

601 

602client = Client()

603 

604# List recent batches

605response = client.batch.list(limit=20)

606 

607for batch in response.batches:

608 status = "complete" if batch.state.num_pending == 0 else "processing"

609 print(f"{batch.name} ({batch.batch_id}): {status}")

610```

611 

612### Check individual request status612### Check individual request status

613 613 

614For detailed tracking, you can inspect the metadata for each request in a batch. This shows the status, timing, and other details for individual requests. This endpoint supports the same `limit` and `pagination_token` parameters for paginating through large batches.614For detailed tracking, you can inspect the metadata for each request in a batch. This shows the status, timing, and other details for individual requests. This endpoint supports the same `limit` and `pagination_token` parameters for paginating through large batches.

618 -H "Authorization: Bearer $XAI_API_KEY"618 -H "Authorization: Bearer $XAI_API_KEY"

619```619```

620 620 

621```pythonXAI

622from xai_sdk import Client

623 

624client = Client()

625 

626# Get metadata for individual requests

627metadata = client.batch.list_batch_requests(batch_id=batch.batch_id)

628 

629for request in metadata.batch_request_metadata:

630 print(f"Request {request.batch_request_id}: {request.state}")

631```

632 

633```javascriptWithoutSDK621```javascriptWithoutSDK

634// Get metadata for individual requests622// Get metadata for individual requests

635const response = await fetch(623const response = await fetch(

643}631}

644```632```

645 633 

634```pythonXAI

635from xai_sdk import Client

636 

637client = Client()

638 

639# Get metadata for individual requests

640metadata = client.batch.list_batch_requests(batch_id=batch.batch_id)

641 

642for request in metadata.batch_request_metadata:

643 print(f"Request {request.batch_request_id}: {request.state}")

644```

645 

646### Track costs646### Track costs

647 647 

648Each batch tracks the total processing cost. Access the cost breakdown after processing to understand your spending. For pricing details, see [Batch API Pricing on the Pricing page](/developers/pricing#batch-api-pricing).648Each batch tracks the total processing cost. Access the cost breakdown after processing to understand your spending. For pricing details, see [Batch API Pricing on the Pricing page](/developers/pricing#batch-api-pricing).

656# Cost is returned in ticks (1e-10 USD) for precision656# Cost is returned in ticks (1e-10 USD) for precision

657```657```

658 658 

659```pythonXAI

660from xai_sdk import Client

661 

662client = Client()

663 

664# Get batch with cost information

665batch = client.batch.get(batch_id=batch.batch_id)

666 

667# Cost is returned in ticks (1e-10 USD) for precision

668total_cost_usd = batch.cost_breakdown.total_cost_usd_ticks / 1e10

669print("Total cost: $%.4f" % total_cost_usd)

670```

671 

672```javascriptWithoutSDK659```javascriptWithoutSDK

673// Get batch with cost information660// Get batch with cost information

674const response = await fetch(661const response = await fetch(

685console.log(\`Total cost: $\${(totalTicks / 1e10).toFixed(4)}\`);672console.log(\`Total cost: $\${(totalTicks / 1e10).toFixed(4)}\`);

686```673```

687 674 

688## Complete example

689 

690This end-to-end example demonstrates a realistic batch workflow: analyzing customer feedback at scale. It creates a batch, submits feedback items for sentiment analysis, waits for processing, and outputs the results. For simplicity, this example doesn't paginate results—see [Step 4](#step-4-retrieve-results) for pagination when processing larger batches.

691 

692```pythonXAI675```pythonXAI

693import time

694from xai_sdk import Client676from xai_sdk import Client

695from xai_sdk.chat import system, user

696 677 

697client = Client()678client = Client()

698 679 

699# Sample dataset: customer feedback to analyze680# Get batch with cost information

700feedback_data = [681batch = client.batch.get(batch_id=batch.batch_id)

701 {"id": "fb_001", "text": "Absolutely love this product! Best purchase ever."},

702 {"id": "fb_002", "text": "Delivery was late and the packaging was damaged."},

703 {"id": "fb_003", "text": "Works fine, nothing special to report."},

704 {"id": "fb_004", "text": "Customer support was incredibly helpful!"},

705 {"id": "fb_005", "text": "The app keeps crashing on my phone."},

706]

707 

708# Step 1: Create a batch

709print("Creating batch...")

710batch = client.batch.create(batch_name="feedback_sentiment_analysis")

711print(f"Batch created: {batch.batch_id}")

712 

713# Step 2: Build and add requests

714print("\\nAdding requests...")

715batch_requests = []

716for item in feedback_data:

717 chat = client.chat.create(

718 model="grok-4.3",

719 batch_request_id=item["id"],

720 )

721 chat.append(system(

722 "Analyze the sentiment of the customer feedback. "

723 "Respond with exactly one word: positive, negative, or neutral."

724 ))

725 chat.append(user(item["text"]))

726 batch_requests.append(chat)

727 

728client.batch.add(batch_id=batch.batch_id, batch_requests=batch_requests)

729print(f"Added {len(batch_requests)} requests")

730 

731# Step 3: Wait for completion

732print("\\nProcessing...")

733while True:

734 batch = client.batch.get(batch_id=batch.batch_id)

735 pending = batch.state.num_pending

736 completed = batch.state.num_success + batch.state.num_error

737

738 print(f" {completed}/{batch.state.num_requests} complete")

739

740 if pending == 0:

741 break

742 time.sleep(2)

743 

744# Step 4: Retrieve and display results

745print("\\n--- Results ---")

746results = client.batch.list_batch_results(batch_id=batch.batch_id)

747 

748# Create a lookup for original feedback text

749feedback_lookup = {item["id"]: item["text"] for item in feedback_data}

750 682 

751for result in results.succeeded:683# Cost is returned in ticks (1e-10 USD) for precision

752 original_text = feedback_lookup.get(result.batch_request_id, "")684total_cost_usd = batch.cost_breakdown.total_cost_usd_ticks / 1e10

753 sentiment = result.response.content.strip().lower()685print("Total cost: $%.4f" % total_cost_usd)

754 print(f"[{sentiment.upper()}] {original_text[:50]}...")686```

755 687 

756# Report any failures688## Complete example

757if results.failed:

758 print("\\n--- Errors ---")

759 for result in results.failed:

760 print(f"[{result.batch_request_id}] {result.error_message}")

761 689 

762# Display cost690This end-to-end example demonstrates a realistic batch workflow: analyzing customer feedback at scale. It creates a batch, submits feedback items for sentiment analysis, waits for processing, and outputs the results. For simplicity, this example doesn't paginate results—see [Step 4](#step-4-retrieve-results) for pagination when processing larger batches.

763cost_usd = batch.cost_breakdown.total_cost_usd_ticks / 1e10

764print("\\nTotal cost: $%.4f" % cost_usd)

765```

766 691 

767```javascriptWithoutSDK692```javascriptWithoutSDK

768const BASE_URL = "https://api.x.ai/v1";693const BASE_URL = "https://api.x.ai/v1";

860}, 2000);785}, 2000);

861```786```

862 787 

788```pythonXAI

789import time

790from xai_sdk import Client

791from xai_sdk.chat import system, user

792 

793client = Client()

794 

795# Sample dataset: customer feedback to analyze

796feedback_data = [

797 {"id": "fb_001", "text": "Absolutely love this product! Best purchase ever."},

798 {"id": "fb_002", "text": "Delivery was late and the packaging was damaged."},

799 {"id": "fb_003", "text": "Works fine, nothing special to report."},

800 {"id": "fb_004", "text": "Customer support was incredibly helpful!"},

801 {"id": "fb_005", "text": "The app keeps crashing on my phone."},

802]

803 

804# Step 1: Create a batch

805print("Creating batch...")

806batch = client.batch.create(batch_name="feedback_sentiment_analysis")

807print(f"Batch created: {batch.batch_id}")

808 

809# Step 2: Build and add requests

810print("\\nAdding requests...")

811batch_requests = []

812for item in feedback_data:

813 chat = client.chat.create(

814 model="grok-4.3",

815 batch_request_id=item["id"],

816 )

817 chat.append(system(

818 "Analyze the sentiment of the customer feedback. "

819 "Respond with exactly one word: positive, negative, or neutral."

820 ))

821 chat.append(user(item["text"]))

822 batch_requests.append(chat)

823 

824client.batch.add(batch_id=batch.batch_id, batch_requests=batch_requests)

825print(f"Added {len(batch_requests)} requests")

826 

827# Step 3: Wait for completion

828print("\\nProcessing...")

829while True:

830 batch = client.batch.get(batch_id=batch.batch_id)

831 pending = batch.state.num_pending

832 completed = batch.state.num_success + batch.state.num_error

833

834 print(f" {completed}/{batch.state.num_requests} complete")

835

836 if pending == 0:

837 break

838 time.sleep(2)

839 

840# Step 4: Retrieve and display results

841print("\\n--- Results ---")

842results = client.batch.list_batch_results(batch_id=batch.batch_id)

843 

844# Create a lookup for original feedback text

845feedback_lookup = {item["id"]: item["text"] for item in feedback_data}

846 

847for result in results.succeeded:

848 original_text = feedback_lookup.get(result.batch_request_id, "")

849 sentiment = result.response.content.strip().lower()

850 print(f"[{sentiment.upper()}] {original_text[:50]}...")

851 

852# Report any failures

853if results.failed:

854 print("\\n--- Errors ---")

855 for result in results.failed:

856 print(f"[{result.batch_request_id}] {result.error_message}")

857 

858# Display cost

859cost_usd = batch.cost_breakdown.total_cost_usd_ticks / 1e10

860print("\\nTotal cost: $%.4f" % cost_usd)

861```

862 

863## JSONL File Upload863## JSONL File Upload

864 864 

865As an alternative to adding requests via the SDK, you can create batches by uploading a JSONL file. This is useful when generating requests from scripts, pipelines, or external tools.865As an alternative to adding requests via the SDK, you can create batches by uploading a JSONL file. This is useful when generating requests from scripts, pipelines, or external tools.

895 895 

896Upload the file via the [Files API](/developers/files), then create a batch referencing it:896Upload the file via the [Files API](/developers/files), then create a batch referencing it:

897 897 

898```pythonXAI

899from xai_sdk import Client

900 

901client = Client()

902 

903# Upload the JSONL file

904file = client.files.upload(

905 file=open("batch_requests.jsonl", "rb"),

906)

907 

908# Create a batch with the file ID

909batch = client.batch.create(

910 batch_name="sentiment_analysis",

911 input_file_id=file.id,

912)

913print(f"Created batch: {batch.batch_id}")

914```

915 

916```bash898```bash

917# Upload the JSONL file899# Upload the JSONL file

918curl -X POST https://api.x.ai/v1/files \\900curl -X POST https://api.x.ai/v1/files \\

957console.log(\`Created batch: \${batch.batch_id}\`);939console.log(\`Created batch: \${batch.batch_id}\`);

958```940```

959 941 

942```pythonXAI

943from xai_sdk import Client

944 

945client = Client()

946 

947# Upload the JSONL file

948file = client.files.upload(

949 file=open("batch_requests.jsonl", "rb"),

950)

951 

952# Create a batch with the file ID

953batch = client.batch.create(

954 batch_name="sentiment_analysis",

955 input_file_id=file.id,

956)

957print(f"Created batch: {batch.batch_id}")

958```

959 

960The file is processed asynchronously in the background. If any line is invalid, the batch is cancelled with an error message. Monitor progress and retrieve results the same way as inline batches.960The file is processed asynchronously in the background. If any line is invalid, the batch is cancelled with an error message. Monitor progress and retrieve results the same way as inline batches.

961 961 

962File-based batches are sealed after creation — you cannot add more requests via `AddBatchRequests`. Maximum file size is **200 MB** with up to **50,000** requests. Each `custom_id` must be unique within the file.962File-based batches are sealed after creation — you cannot add more requests via `AddBatchRequests`. Maximum file size is **200 MB** with up to **50,000** requests. Each `custom_id` must be unique within the file.

985## Related985## Related

986 986 

987* [API Reference: Batch endpoints](/developers/rest-api-reference/inference/batches#create-a-new-batch)987* [API Reference: Batch endpoints](/developers/rest-api-reference/inference/batches#create-a-new-batch)

988* [gRPC Reference: Batch Management](/developers/grpc-api-reference/batches)

989* [Pricing — Batch API Pricing](/developers/pricing#batch-api-pricing)988* [Pricing — Batch API Pricing](/developers/pricing#batch-api-pricing)

990* [xAI Python SDK](https://github.com/xai-org/xai-sdk-python)989* [xAI Python SDK](https://github.com/xai-org/xai-sdk-python)

29 29 

30Send the conversation you want to compact. The response contains a single compaction item that stands in for the entire prior conversation — you can safely drop the original messages from your client-side state, use the compaction item as the head of your next request, and append your new user turn after it.30Send the conversation you want to compact. The response contains a single compaction item that stands in for the entire prior conversation — you can safely drop the original messages from your client-side state, use the compaction item as the head of your next request, and append your new user turn after it.

31 31 

32```bash customLanguage="bash"

33# Step 1 — compact the long conversation

34curl -s https://api.x.ai/v1/responses/compact \

35 -H "Content-Type: application/json" \

36 -H "Authorization: Bearer $XAI_API_KEY" \

37 -d '{

38 "model": "grok-4.7",

39 "input": [

40 {"role": "system", "content": "You are a concise and knowledgeable science tutor."},

41 {"role": "user", "content": "What is the Higgs boson and why is it important?"},

42 {"role": "assistant", "content": "The Higgs boson is an elementary particle..."},

43 {"role": "user", "content": "How does the Higgs mechanism actually work?"},

44 {"role": "assistant", "content": "The Higgs mechanism works through spontaneous symmetry breaking..."}

45 ]

46 }'

47 

48# Step 2 — continue the conversation using the compacted output

49curl -s https://api.x.ai/v1/responses \

50 -H "Content-Type: application/json" \

51 -H "Authorization: Bearer $XAI_API_KEY" \

52 -d '{

53 "model": "grok-4.7",

54 "input": [

55 {

56 "type": "compaction",

57 "id": "cmp_abc123",

58 "encrypted_content": "<paste encrypted_content from step 1>"

59 },

60 {"role": "user", "content": "Based on our earlier conversation, what gives particles their mass?"}

61 ]

62 }'

63```

64 

65```python customLanguage="pythonXAI"

66import os

67from xai_sdk import Client

68from xai_sdk.chat import system, user

69 

70client = Client(api_key=os.environ["XAI_API_KEY"])

71 

72# Build up a chat normally — system prompt plus a few user/assistant turns.

73# use_encrypted_content=True is recommended for reasoning models so the model's

74# reasoning content from prior turns is preserved through the compaction.

75chat = client.chat.create(model="grok-4.7", use_encrypted_content=True)

76chat.append(system("You are a concise and knowledgeable science tutor."))

77 

78chat.append(user("What is the Higgs boson and why is it important?"))

79chat.append(chat.sample())

80 

81chat.append(user("How does the Higgs mechanism actually work?"))

82chat.append(chat.sample())

83 

84# ... many more turns ...

85 

86# Step 1 — compact the conversation. Pass the chat's accumulated messages

87# straight into compact_context.

88compact = client.chat.compact_context(

89 model="grok-4.7",

90 messages=chat.messages,

91)

92print(f"Compaction ID: {compact.id}")

93print(f"Dropped messages: {compact.dropped_message_count}")

94print(f"Tokens used: {compact.usage.total_tokens}")

95 

96# Step 2 — continue the conversation. chat.append(compact) clears the

97# in-memory message list on the chat object and seeds it with just the

98# compaction blob, so subsequent chat.sample() calls run on top of the

99# compacted context instead of replaying the full prior history.

100chat.append(compact)

101chat.append(user("Based on our earlier conversation, what gives particles their mass?"))

102print(chat.sample().content)

103```

104 

105```python customLanguage="pythonOpenAISDK"32```python customLanguage="pythonOpenAISDK"

106import os33import os

107from openai import OpenAI34from openai import OpenAI

139print(followup.output_text)66print(followup.output_text)

140```67```

141 68 

69```bash customLanguage="bash"

70# Step 1 — compact the long conversation

71curl -s https://api.x.ai/v1/responses/compact \

72 -H "Content-Type: application/json" \

73 -H "Authorization: Bearer $XAI_API_KEY" \

74 -d '{

75 "model": "grok-4.7",

76 "input": [

77 {"role": "system", "content": "You are a concise and knowledgeable science tutor."},

78 {"role": "user", "content": "What is the Higgs boson and why is it important?"},

79 {"role": "assistant", "content": "The Higgs boson is an elementary particle..."},

80 {"role": "user", "content": "How does the Higgs mechanism actually work?"},

81 {"role": "assistant", "content": "The Higgs mechanism works through spontaneous symmetry breaking..."}

82 ]

83 }'

84 

85# Step 2 — continue the conversation using the compacted output

86curl -s https://api.x.ai/v1/responses \

87 -H "Content-Type: application/json" \

88 -H "Authorization: Bearer $XAI_API_KEY" \

89 -d '{

90 "model": "grok-4.7",

91 "input": [

92 {

93 "type": "compaction",

94 "id": "cmp_abc123",

95 "encrypted_content": "<paste encrypted_content from step 1>"

96 },

97 {"role": "user", "content": "Based on our earlier conversation, what gives particles their mass?"}

98 ]

99 }'

100```

101 

142```javascript customLanguage="javascriptOpenAISDK"102```javascript customLanguage="javascriptOpenAISDK"

143import OpenAI from "openai";103import OpenAI from "openai";

144 104 

175console.log(followup.output_text);135console.log(followup.output_text);

176```136```

177 137 

138```python customLanguage="pythonXAI"

139import os

140from xai_sdk import Client

141from xai_sdk.chat import system, user

142 

143client = Client(api_key=os.environ["XAI_API_KEY"])

144 

145# Build up a chat normally — system prompt plus a few user/assistant turns.

146# use_encrypted_content=True is recommended for reasoning models so the model's

147# reasoning content from prior turns is preserved through the compaction.

148chat = client.chat.create(model="grok-4.7", use_encrypted_content=True)

149chat.append(system("You are a concise and knowledgeable science tutor."))

150 

151chat.append(user("What is the Higgs boson and why is it important?"))

152chat.append(chat.sample())

153 

154chat.append(user("How does the Higgs mechanism actually work?"))

155chat.append(chat.sample())

156 

157# ... many more turns ...

158 

159# Step 1 — compact the conversation. Pass the chat's accumulated messages

160# straight into compact_context.

161compact = client.chat.compact_context(

162 model="grok-4.7",

163 messages=chat.messages,

164)

165print(f"Compaction ID: {compact.id}")

166print(f"Dropped messages: {compact.dropped_message_count}")

167print(f"Tokens used: {compact.usage.total_tokens}")

168 

169# Step 2 — continue the conversation. chat.append(compact) clears the

170# in-memory message list on the chat object and seeds it with just the

171# compaction blob, so subsequent chat.sample() calls run on top of the

172# compacted context instead of replaying the full prior history.

173chat.append(compact)

174chat.append(user("Based on our earlier conversation, what gives particles their mass?"))

175print(chat.sample().content)

176```

177 

178The xAI SDK also exposes an `AsyncClient` with `await client.chat.compact_context(...)` and `await chat.sample()` for the same flow under `asyncio`.178The xAI SDK also exposes an `AsyncClient` with `await client.chat.compact_context(...)` and `await chat.sample()` for the same flow under `asyncio`.

179 179 

180### Response shape180### Response shape

26 26 

27A code example is provided below, where we retry retrieving the result until it has been processed:27A code example is provided below, where we retry retrieving the result until it has been processed:

28 28 

29```pythonXAI

30import os

31from datetime import timedelta

32 

33from xai_sdk import Client

34from xai_sdk.chat import user, system

35 

36client = Client(api_key=os.getenv('XAI_API_KEY'))

37 

38chat = client.chat.create(

39 model="grok-4.7",

40 messages=[system("You are Zaphod Beeblebrox.")]

41)

42chat.append(user("126/3=?"))

43 

44# Poll the result every 10 seconds for a maximum of 10 minutes

45 

46response = chat.defer(

47 timeout=timedelta(minutes=10), interval=timedelta(seconds=10)

48)

49 

50# Print the result when it is ready

51 

52print(response.content)

53```

54 

55```pythonRequests29```pythonRequests

56import json30import json

57import os31import os

162-H "Authorization: Bearer $XAI_API_KEY"136-H "Authorization: Bearer $XAI_API_KEY"

163```137```

164 138 

139```pythonXAI

140import os

141from datetime import timedelta

142 

143from xai_sdk import Client

144from xai_sdk.chat import user, system

145 

146client = Client(api_key=os.getenv('XAI_API_KEY'))

147 

148chat = client.chat.create(

149 model="grok-4.7",

150 messages=[system("You are Zaphod Beeblebrox.")]

151)

152chat.append(user("126/3=?"))

153 

154# Poll the result every 10 seconds for a maximum of 10 minutes

155 

156response = chat.defer(

157 timeout=timedelta(minutes=10), interval=timedelta(seconds=10)

158)

159 

160# Print the result when it is ready

161 

162print(response.content)

163```

164 

165The response body will be the same as what you would expect with non-deferred chat completions:165The response body will be the same as what you would expect with non-deferred chat completions:

166 166 

167```json167```json

34 34 

35Include your client certificate and private key with every request. Here are examples:35Include your client certificate and private key with every request. Here are examples:

36 36 

37```bash

38curl https://mtls.api.x.ai/v1/chat/completions \\

39 --cert /path/to/client-cert.pem \\

40 --key /path/to/client-key.pem \\

41 -H "Content-Type: application/json" \\

42 -H "Authorization: Bearer $XAI_API_KEY" \\

43 -d '{

44 "messages": [

45 {

46 "role": "user",

47 "content": "Hello, world!"

48 }

49 ],

50 "model": "grok-4.7",

51 "stream": false

52 }'

53```

54 

55```pythonOpenAISDK37```pythonOpenAISDK

56import os38import os

57import httpx39import httpx

77print(completion.choices[0].message.content)59print(completion.choices[0].message.content)

78```60```

79 61 

62```bash

63curl https://mtls.api.x.ai/v1/chat/completions \\

64 --cert /path/to/client-cert.pem \\

65 --key /path/to/client-key.pem \\

66 -H "Content-Type: application/json" \\

67 -H "Authorization: Bearer $XAI_API_KEY" \\

68 -d '{

69 "messages": [

70 {

71 "role": "user",

72 "content": "Hello, world!"

73 }

74 ],

75 "model": "grok-4.7",

76 "stream": false

77 }'

78```

79 

80```javascriptOpenAISDK80```javascriptOpenAISDK

81import OpenAI from 'openai';81import OpenAI from 'openai';

82import https from 'https';82import https from 'https';

23 23 

24Pass `service_tier: "priority"` in your request body. The response includes a `service_tier` field confirming which tier was used.24Pass `service_tier: "priority"` in your request body. The response includes a `service_tier` field confirming which tier was used.

25 25 

26```bash customLanguage="bash"

27curl https://api.x.ai/v1/responses \

28 -H "Authorization: Bearer $XAI_API_KEY" \

29 -H "Content-Type: application/json" \

30 -d '{

31 "model": "grok-4.7",

32 "input": "Explain the Riemann hypothesis in one paragraph.",

33 "service_tier": "priority"

34 }'

35```

36 

37```python customLanguage="pythonXAI"

38import os

39 

40from xai_sdk import Client

41from xai_sdk.chat import user

42 

43client = Client(api_key=os.getenv("XAI_API_KEY"))

44 

45chat = client.chat.create(

46 model="grok-4.7",

47 service_tier="priority",

48)

49chat.append(user("Explain the Riemann hypothesis in one paragraph."))

50 

51response = chat.sample()

52 

53print(response.content)

54print(f"Tier used: {response.service_tier}")

55```

56 

57```python customLanguage="pythonOpenAISDK"26```python customLanguage="pythonOpenAISDK"

58import os27import os

59from openai import OpenAI28from openai import OpenAI

73print(f"Tier used: {response.service_tier}")42print(f"Tier used: {response.service_tier}")

74```43```

75 44 

45```bash customLanguage="bash"

46curl https://api.x.ai/v1/responses \

47 -H "Authorization: Bearer $XAI_API_KEY" \

48 -H "Content-Type: application/json" \

49 -d '{

50 "model": "grok-4.7",

51 "input": "Explain the Riemann hypothesis in one paragraph.",

52 "service_tier": "priority"

53 }'

54```

55 

76```javascript customLanguage="javascriptOpenAISDK"56```javascript customLanguage="javascriptOpenAISDK"

77import OpenAI from "openai";57import OpenAI from "openai";

78 58 

91console.log(`Tier used: ${response.service_tier}`);71console.log(`Tier used: ${response.service_tier}`);

92```72```

93 73 

74```python customLanguage="pythonXAI"

75import os

76 

77from xai_sdk import Client

78from xai_sdk.chat import user

79 

80client = Client(api_key=os.getenv("XAI_API_KEY"))

81 

82chat = client.chat.create(

83 model="grok-4.7",

84 service_tier="priority",

85)

86chat.append(user("Explain the Riemann hypothesis in one paragraph."))

87 

88response = chat.sample()

89 

90print(response.content)

91print(f"Tier used: {response.service_tier}")

92```

93 

94The response includes `"service_tier": "priority"` when the request was served at the priority tier, or `"service_tier": "default"` if it was served at the default tier instead. You are only billed at the priority rate when the response confirms `"priority"`.94The response includes `"service_tier": "priority"` when the request was served at the priority tier, or `"service_tier": "default"` if it was served at the default tier instead. You are only billed at the priority rate when the response confirms `"priority"`.

95 95 

96```json customLanguage="json"96```json customLanguage="json"

6 6 

7The `x-grok-conv-id` HTTP header routes requests with the same conversation ID to the same server. Since cache entries are stored per-server, this maximizes your cache hit rate.7The `x-grok-conv-id` HTTP header routes requests with the same conversation ID to the same server. Since cache entries are stored per-server, this maximizes your cache hit rate.

8 8 

9```bash customLanguage="bash"

10curl https://api.x.ai/v1/chat/completions \

11 -H "Content-Type: application/json" \

12 -H "Authorization: Bearer $XAI_API_KEY" \

13 -H "x-grok-conv-id: conv_abc123" \

14 -d '{

15 "model": "grok-4.7",

16 "messages": [

17 {"role": "system", "content": "You are Grok, a helpful and truthful AI assistant built by xAI."},

18 {"role": "user", "content": "What is prompt caching?"}

19 ]

20 }'

21```

22 

23```python customLanguage="pythonOpenAISDK"9```python customLanguage="pythonOpenAISDK"

24from openai import OpenAI10from openai import OpenAI

25 11 

43print(f"Cached tokens: {response.usage.prompt_tokens_details.cached_tokens}")29print(f"Cached tokens: {response.usage.prompt_tokens_details.cached_tokens}")

44```30```

45 31 

32```bash customLanguage="bash"

33curl https://api.x.ai/v1/chat/completions \

34 -H "Content-Type: application/json" \

35 -H "Authorization: Bearer $XAI_API_KEY" \

36 -H "x-grok-conv-id: conv_abc123" \

37 -d '{

38 "model": "grok-4.7",

39 "messages": [

40 {"role": "system", "content": "You are Grok, a helpful and truthful AI assistant built by xAI."},

41 {"role": "user", "content": "What is prompt caching?"}

42 ]

43 }'

44```

45 

46```javascript customLanguage="javascriptOpenAISDK"46```javascript customLanguage="javascriptOpenAISDK"

47import OpenAI from 'openai';47import OpenAI from 'openai';

48 48 

80 80 

81For the Responses API, use the `prompt_cache_key` field directly in the request body. It functions identically to setting `x-grok-conv-id` — it routes requests to the same server for cache reuse.81For the Responses API, use the `prompt_cache_key` field directly in the request body. It functions identically to setting `x-grok-conv-id` — it routes requests to the same server for cache reuse.

82 82 

83```bash customLanguage="bash"83```javascript customLanguage="javascriptAISDK"

84curl https://api.x.ai/v1/responses \84import { xai } from '@ai-sdk/xai';

85 -H "Content-Type: application/json" \85import { generateText } from 'ai';

86 -H "Authorization: Bearer $XAI_API_KEY" \86 

87 -d '{87const { text, usage } = await generateText({

88 "model": "grok-4.7",88 model: xai.responses('grok-4.7'),

89 "input": "What is prompt caching?",89 prompt: 'What is prompt caching?',

90 "prompt_cache_key": "b79ad29b-b3f9-463c-bca6-041d5058d366"90 providerOptions: {

91 }'91 xai: {

92 promptCacheKey: 'b79ad29b-b3f9-463c-bca6-041d5058d366',

93 },

94 },

95});

96 

97console.log(text);

98console.log(`Total tokens: ${usage.totalTokens}`);

92```99```

93 100 

94```python customLanguage="pythonOpenAISDK"101```python customLanguage="pythonOpenAISDK"

111print(f"Cached tokens: {response.usage.input_tokens_details.cached_tokens}")118print(f"Cached tokens: {response.usage.input_tokens_details.cached_tokens}")

112```119```

113 120 

121```bash customLanguage="bash"

122curl https://api.x.ai/v1/responses \

123 -H "Content-Type: application/json" \

124 -H "Authorization: Bearer $XAI_API_KEY" \

125 -d '{

126 "model": "grok-4.7",

127 "input": "What is prompt caching?",

128 "prompt_cache_key": "b79ad29b-b3f9-463c-bca6-041d5058d366"

129 }'

130```

131 

114```javascript customLanguage="javascriptOpenAISDK"132```javascript customLanguage="javascriptOpenAISDK"

115import OpenAI from 'openai';133import OpenAI from 'openai';

116 134 

132);150);

133```151```

134 152 

135```javascript customLanguage="javascriptAISDK"

136import { xai } from '@ai-sdk/xai';

137import { generateText } from 'ai';

138 

139const { text, usage } = await generateText({

140 model: xai.responses('grok-4.7'),

141 prompt: 'What is prompt caching?',

142 providerOptions: {

143 xai: {

144 promptCacheKey: 'b79ad29b-b3f9-463c-bca6-041d5058d366',

145 },

146 },

147});

148 

149console.log(text);

150console.log(`Total tokens: ${usage.totalTokens}`);

151```

152 

153## Set `x-grok-conv-id` metadata (gRPC API)153## Set `x-grok-conv-id` metadata (gRPC API)

154 154 

155For the gRPC API using the xAI SDK, pass `x-grok-conv-id` as gRPC metadata to enable sticky routing for cache reuse.155For the gRPC API using the xAI SDK, pass `x-grok-conv-id` as gRPC metadata to enable sticky routing for cache reuse.

17 17 

18The prompt prefix is identical to the previous request, with only a new user message appended:18The prompt prefix is identical to the previous request, with only a new user message appended:

19 19 

20```bash customLanguage="bash" addedLines="26"

21# Turn 1: Initial request (establishes the cache)

22curl https://api.x.ai/v1/chat/completions \

23 -H "Content-Type: application/json" \

24 -H "Authorization: Bearer $XAI_API_KEY" \

25 -H "x-grok-conv-id: conv_abc123" \

26 -d '{

27 "model": "grok-4.7",

28 "messages": [

29 {"role": "system", "content": "You are Grok, a helpful and truthful AI assistant built by xAI."},

30 {"role": "user", "content": "What is prompt caching?"},

31 {"role": "assistant", "content": "Prompt caching stores KV pairs from unchanged prompt prefixes so they can be reused on subsequent requests. This makes responses faster and cheaper."}

32 ]

33 }'

34 

35# Turn 2: Cache HIT — exact prefix preserved, new message appended

36curl https://api.x.ai/v1/chat/completions \

37 -H "Content-Type: application/json" \

38 -H "Authorization: Bearer $XAI_API_KEY" \

39 -H "x-grok-conv-id: conv_abc123" \

40 -d '{

41 "model": "grok-4.7",

42 "messages": [

43 {"role": "system", "content": "You are Grok, a helpful and truthful AI assistant built by xAI."},

44 {"role": "user", "content": "What is prompt caching?"},

45 {"role": "assistant", "content": "Prompt caching stores KV pairs from unchanged prompt prefixes so they can be reused on subsequent requests. This makes responses faster and cheaper."},

46 {"role": "user", "content": "Show me a code example."}

47 ]

48 }'

49```

50 

51```python customLanguage="pythonOpenAISDK"20```python customLanguage="pythonOpenAISDK"

52from openai import OpenAI21from openai import OpenAI

53 22 

83print(f"Turn 2 — Cached tokens: {response.usage.prompt_tokens_details.cached_tokens}")52print(f"Turn 2 — Cached tokens: {response.usage.prompt_tokens_details.cached_tokens}")

84```53```

85 54 

55```bash customLanguage="bash" addedLines="26"

56# Turn 1: Initial request (establishes the cache)

57curl https://api.x.ai/v1/chat/completions \

58 -H "Content-Type: application/json" \

59 -H "Authorization: Bearer $XAI_API_KEY" \

60 -H "x-grok-conv-id: conv_abc123" \

61 -d '{

62 "model": "grok-4.7",

63 "messages": [

64 {"role": "system", "content": "You are Grok, a helpful and truthful AI assistant built by xAI."},

65 {"role": "user", "content": "What is prompt caching?"},

66 {"role": "assistant", "content": "Prompt caching stores KV pairs from unchanged prompt prefixes so they can be reused on subsequent requests. This makes responses faster and cheaper."}

67 ]

68 }'

69 

70# Turn 2: Cache HIT — exact prefix preserved, new message appended

71curl https://api.x.ai/v1/chat/completions \

72 -H "Content-Type: application/json" \

73 -H "Authorization: Bearer $XAI_API_KEY" \

74 -H "x-grok-conv-id: conv_abc123" \

75 -d '{

76 "model": "grok-4.7",

77 "messages": [

78 {"role": "system", "content": "You are Grok, a helpful and truthful AI assistant built by xAI."},

79 {"role": "user", "content": "What is prompt caching?"},

80 {"role": "assistant", "content": "Prompt caching stores KV pairs from unchanged prompt prefixes so they can be reused on subsequent requests. This makes responses faster and cheaper."},

81 {"role": "user", "content": "Show me a code example."}

82 ]

83 }'

84```

85 

86```javascript customLanguage="javascriptOpenAISDK"86```javascript customLanguage="javascriptOpenAISDK"

87import OpenAI from 'openai';87import OpenAI from 'openai';

88 88 

31console.log(text);31console.log(text);

32```32```

33 33 

34```python customLanguage="pythonXAI"

35import os

36 

37from xai_sdk import Client

38from xai_sdk.chat import user

39 

40client = Client(

41 api_key=os.getenv("XAI_API_KEY"),

42 api_host="us.api.x.ai",

43)

44 

45chat = client.chat.create(model="grok-4.7")

46chat.append(user("Explain latency versus throughput in two sentences."))

47 

48print(chat.sample().content)

49```

50 

51```python customLanguage="pythonOpenAISDK"34```python customLanguage="pythonOpenAISDK"

52import os35import os

53from openai import OpenAI36from openai import OpenAI

91 }'74 }'

92```75```

93 76 

77```python customLanguage="pythonXAI"

78import os

79 

80from xai_sdk import Client

81from xai_sdk.chat import user

82 

83client = Client(

84 api_key=os.getenv("XAI_API_KEY"),

85 api_host="us.api.x.ai",

86)

87 

88chat = client.chat.create(model="grok-4.7")

89chat.append(user("Explain latency versus throughput in two sentences."))

90 

91print(chat.sample().content)

92```

93 

94### Model availability94### Model availability

95 95 

96`grok-4.7` and `grok-4.6` are currently the only models available on the US endpoint; the [models page in the console](https://console.x.ai/team/default/models?cluster=us-central-1\&utm_source=docs\&utm_medium=referral\&utm_campaign=developers-advanced-api-usage-regions\&utm_content=models) and `GET https://us.api.x.ai/v1/models` always show the current list. Requesting a model that is not on that list, including `grok-latest`, fails with `404 Not Found`:96`grok-4.7` and `grok-4.6` are currently the only models available on the US endpoint; the [models page in the console](https://console.x.ai/team/default/models?cluster=us-central-1\&utm_source=docs\&utm_medium=referral\&utm_campaign=developers-advanced-api-usage-regions\&utm_content=models) and `GET https://us.api.x.ai/v1/models` always show the current list. Requesting a model that is not on that list, including `grok-latest`, fails with `404 Not Found`:

57}57}

58```58```

59 59 

60```bash customLanguage="bash"

61curl https://api.x.ai/v1/responses \

62 -H "Authorization: Bearer $XAI_API_KEY" \

63 -H "Content-Type: application/json" \

64 -d '{

65 "model": "grok-4.7",

66 "input": "Say hello"

67 }' | jq '.usage.cost_in_usd_ticks'

68```

69 

70```python customLanguage="pythonOpenAISDK"60```python customLanguage="pythonOpenAISDK"

71import os61import os

72from openai import OpenAI62from openai import OpenAI

87print(f"Cost: ${cost_usd:.6f}")77print(f"Cost: ${cost_usd:.6f}")

88```78```

89 79 

80```bash customLanguage="bash"

81curl https://api.x.ai/v1/responses \

82 -H "Authorization: Bearer $XAI_API_KEY" \

83 -H "Content-Type: application/json" \

84 -d '{

85 "model": "grok-4.7",

86 "input": "Say hello"

87 }' | jq '.usage.cost_in_usd_ticks'

88```

89 

90```javascript customLanguage="javascriptOpenAISDK"90```javascript customLanguage="javascriptOpenAISDK"

91import OpenAI from "openai";91import OpenAI from "openai";

92 92 

115 115 

116When using the OpenAI SDK or the REST API, set `stream_options: { include_usage: true }` on the request. Cost is only included in the final chunk (with empty `choices`); intermediate chunks do not contain usage data.116When using the OpenAI SDK or the REST API, set `stream_options: { include_usage: true }` on the request. Cost is only included in the final chunk (with empty `choices`); intermediate chunks do not contain usage data.

117 117 

118```python customLanguage="pythonXAI"

119import os

120from xai_sdk import Client

121from xai_sdk.chat import user

122 

123client = Client(api_key=os.getenv("XAI_API_KEY"))

124 

125chat = client.chat.create(

126 model="grok-4.7",

127 messages=[user("Tell me a joke")],

128)

129 

130for response, chunk in chat.stream():

131 print(chunk.content, end="", flush=True)

132print()

133 

134# After the stream completes, cost is on the final response.

135print(f"Cost: ${response.cost_usd:.6f}")

136```

137 

138```python customLanguage="pythonOpenAISDK"118```python customLanguage="pythonOpenAISDK"

139import os119import os

140from openai import OpenAI120from openai import OpenAI

159 print(chunk.choices[0].delta.content or "", end="", flush=True)139 print(chunk.choices[0].delta.content or "", end="", flush=True)

160```140```

161 141 

162## Tracking cost across a conversation

163 

164`cost_in_usd_ticks` is per-request; it does not accumulate across turns. In a multi-turn conversation, sum the costs yourself:

165 

166```python customLanguage="pythonXAI"142```python customLanguage="pythonXAI"

167import os143import os

168from xai_sdk import Client144from xai_sdk import Client

169from xai_sdk.chat import system, user145from xai_sdk.chat import user

170 146 

171client = Client(api_key=os.getenv("XAI_API_KEY"))147client = Client(api_key=os.getenv("XAI_API_KEY"))

172 148 

173chat = client.chat.create(149chat = client.chat.create(

174 model="grok-4.7",150 model="grok-4.7",

175 messages=[system("You are a helpful assistant.")],151 messages=[user("Tell me a joke")],

176)152)

177 153 

178total_cost_usd = 0.0154for response, chunk in chat.stream():

179while True:155 print(chunk.content, end="", flush=True)

180 prompt = input("You: ")156print()

181 if prompt.lower() == "exit":

182 break

183 157 

184 chat.append(user(prompt))158# After the stream completes, cost is on the final response.

185 response = chat.sample()159print(f"Cost: ${response.cost_usd:.6f}")

186 print(f"Grok: {response.content}")160```

187 chat.append(response)

188 161 

189 total_cost_usd += response.cost_usd or 0.0162## Tracking cost across a conversation

190 print(f" (this turn: ${response.cost_usd or 0:.6f})")

191 163 

192print(f"Total session cost: ${total_cost_usd:.4f}")164`cost_in_usd_ticks` is per-request; it does not accumulate across turns. In a multi-turn conversation, sum the costs yourself:

193```

194 165 

195```python customLanguage="pythonOpenAISDK"166```python customLanguage="pythonOpenAISDK"

196import os167import os

227print(f"Total session cost: ${total_cost_usd:.4f}")198print(f"Total session cost: ${total_cost_usd:.4f}")

228```199```

229 200 

230## Server-side tools

231 

232When a request uses server-side tools (web search, X search, code execution), the model may make multiple internal calls before returning a final answer. The returned `cost_in_usd_ticks` covers all token costs and all tool invocations from that request in a single value. No separate accumulation needed.

233 

234```python customLanguage="pythonXAI"201```python customLanguage="pythonXAI"

235import os202import os

236from xai_sdk import Client203from xai_sdk import Client

237from xai_sdk.chat import user204from xai_sdk.chat import system, user

238from xai_sdk.tools import web_search, x_search

239 205 

240client = Client(api_key=os.getenv("XAI_API_KEY"))206client = Client(api_key=os.getenv("XAI_API_KEY"))

241 207 

242chat = client.chat.create(208chat = client.chat.create(

243 model="grok-4.7",209 model="grok-4.7",

244 tools=[web_search(), x_search()],210 messages=[system("You are a helpful assistant.")],

245)211)

246chat.append(user("What are people saying about xAI's latest announcement?"))

247 212 

248response = chat.sample()213total_cost_usd = 0.0

249print(response.content)214while True:

215 prompt = input("You: ")

216 if prompt.lower() == "exit":

217 break

250 218 

251# Shows which server-side tools were invoked and how many times.219 chat.append(user(prompt))

252print(f"Tools used: {response.server_side_tool_usage}")220 response = chat.sample()

253# Cost covers all model decodes + every tool call in the agentic loop.221 print(f"Grok: {response.content}")

254print(f"Cost: ${response.cost_usd:.4f}")222 chat.append(response)

223 

224 total_cost_usd += response.cost_usd or 0.0

225 print(f" (this turn: ${response.cost_usd or 0:.6f})")

226 

227print(f"Total session cost: ${total_cost_usd:.4f}")

255```228```

256 229 

230## Server-side tools

231 

232When a request uses server-side tools (web search, X search, code execution), the model may make multiple internal calls before returning a final answer. The returned `cost_in_usd_ticks` covers all token costs and all tool invocations from that request in a single value. No separate accumulation needed.

233 

257```python customLanguage="pythonOpenAISDK"234```python customLanguage="pythonOpenAISDK"

258import os235import os

259from openai import OpenAI236from openai import OpenAI

290 }' | jq '{tools_used: .usage.num_server_side_tools_used, cost_in_usd_ticks: .usage.cost_in_usd_ticks}'267 }' | jq '{tools_used: .usage.num_server_side_tools_used, cost_in_usd_ticks: .usage.cost_in_usd_ticks}'

291```268```

292 269 

270```python customLanguage="pythonXAI"

271import os

272from xai_sdk import Client

273from xai_sdk.chat import user

274from xai_sdk.tools import web_search, x_search

275 

276client = Client(api_key=os.getenv("XAI_API_KEY"))

277 

278chat = client.chat.create(

279 model="grok-4.7",

280 tools=[web_search(), x_search()],

281)

282chat.append(user("What are people saying about xAI's latest announcement?"))

283 

284response = chat.sample()

285print(response.content)

286 

287# Shows which server-side tools were invoked and how many times.

288print(f"Tools used: {response.server_side_tool_usage}")

289# Cost covers all model decodes + every tool call in the agentic loop.

290print(f"Cost: ${response.cost_usd:.4f}")

291```

292 

293## Image and video generation293## Image and video generation

294 294 

295Image and video responses include the same `cost_in_usd_ticks` field in their `usage` object:295Image and video responses include the same `cost_in_usd_ticks` field in their `usage` object:

296 296 

297```python customLanguage="pythonOpenAISDK"

298import os

299from openai import OpenAI

300 

301client = OpenAI(

302 api_key=os.getenv("XAI_API_KEY"),

303 base_url="https://api.x.ai/v1",

304)

305 

306response = client.images.generate(

307 model="grok-imagine-image-2.0",

308 prompt="A cat on a rocket",

309)

310 

311cost_ticks = response.usage.cost_in_usd_ticks

312print(f"Image cost: ${cost_ticks / 1e10:.4f}")

313```

314 

297```bash customLanguage="bash"315```bash customLanguage="bash"

298# Image generation316# Image generation

299curl https://api.x.ai/v1/images/generations \317curl https://api.x.ai/v1/images/generations \

327print(f"Video cost: ${video.cost_usd:.4f}")345print(f"Video cost: ${video.cost_usd:.4f}")

328```346```

329 347 

330```python customLanguage="pythonOpenAISDK"

331import os

332from openai import OpenAI

333 

334client = OpenAI(

335 api_key=os.getenv("XAI_API_KEY"),

336 base_url="https://api.x.ai/v1",

337)

338 

339response = client.images.generate(

340 model="grok-imagine-image-2.0",

341 prompt="A cat on a rocket",

342)

343 

344cost_ticks = response.usage.cost_in_usd_ticks

345print(f"Image cost: ${cost_ticks / 1e10:.4f}")

346```

347 

348## Batch API348## Batch API

349 349 

350Batch results include per-request costs. You can sum them to get the total batch cost, or read the `cost_breakdown` on the batch object itself. See [Batch API](/developers/advanced-api-usage/batch-api) for details.350Batch results include per-request costs. You can sum them to get the total batch cost, or read the `cost_breakdown` on the batch object itself. See [Batch API](/developers/advanced-api-usage/batch-api) for details.

29 }'29 }'

30```30```

31 31 

32`safety_identifier` is accepted on [Chat Completions](/developers/rest-api-reference/inference/chat-completions), the [Responses API](/developers/rest-api-reference/inference/responses) (including streaming and [WebSocket mode](/developers/advanced-api-usage/websocket-mode)), [deferred chat completions](/developers/advanced-api-usage/deferred-chat-completions), the [Batch API](/developers/advanced-api-usage/batch-api), and the [gRPC `GetCompletionsRequest`](/developers/grpc-api-reference/chat). The legacy `user` field is still accepted for compatibility; `safety_identifier` is the documented field going forward.32`safety_identifier` is accepted on [Chat Completions](/developers/rest-api-reference/inference/chat-completions), the [Responses API](/developers/rest-api-reference/inference/responses) (including streaming and [WebSocket mode](/developers/advanced-api-usage/websocket-mode)), [deferred chat completions](/developers/advanced-api-usage/deferred-chat-completions), the [Batch API](/developers/advanced-api-usage/batch-api), and the gRPC `GetCompletionsRequest`. The legacy `user` field is still accepted for compatibility; `safety_identifier` is the documented field going forward.

33 33 

34If you run a gateway or platform that routes many end users through one API key (OpenRouter, Vercel AI Gateway, and similar), set `safety_identifier` per end user so an abuse report can be scoped to that user rather than to your key.34If you run a gateway or platform that routes many end users through one API key (OpenRouter, Vercel AI Gateway, and similar), set `safety_identifier` per end user so an abuse report can be scoped to that user rather than to your key.

35 35 

55| **[Collections API](/developers/files/collections)** | Stores your documents on SpaceXAI so Grok can search and retrieve from them (RAG). Those documents must be stored to be retrieved, which ZDR does not allow. Existing collections can still be viewed and deleted. |55| **[Collections API](/developers/files/collections)** | Stores your documents on SpaceXAI so Grok can search and retrieve from them (RAG). Those documents must be stored to be retrieved, which ZDR does not allow. Existing collections can still be viewed and deleted. |

56| **[Batch API](/developers/advanced-api-usage/batch-api)** | Processes large sets of requests asynchronously at a discounted price. Batched requests are stored until they are processed, so the Batch API is unavailable under ZDR. |56| **[Batch API](/developers/advanced-api-usage/batch-api)** | Processes large sets of requests asynchronously at a discounted price. Batched requests are stored until they are processed, so the Batch API is unavailable under ZDR. |

57| **[Deferred completions](/developers/advanced-api-usage/deferred-chat-completions)** | Return a request ID right away and let you fetch the result later instead of holding the connection open. That result is stored until you retrieve it, which ZDR does not permit. Concurrent client-side requests (for example via `AsyncClient`) are unaffected. |57| **[Deferred completions](/developers/advanced-api-usage/deferred-chat-completions)** | Return a request ID right away and let you fetch the result later instead of holding the connection open. That result is stored until you retrieve it, which ZDR does not permit. Concurrent client-side requests (for example via `AsyncClient`) are unaffected. |

58| **Stored [image](/developers/model-capabilities/images/generation) and [video](/developers/model-capabilities/video/generation) outputs** | Image and video generation can have SpaceXAI host the generated media and return it as a `file_id` or URL. Under ZDR there is no SpaceXAI-side output storage: no `file_id` inputs, no stored outputs, and no URL-format image results. For images, return base64 only. For video, you must supply your own `output.upload_url` (SpaceXAI uploads the result to your URL). Image generation is also limited to `grok-imagine` models and excludes agentic image generation. Video generation in the Console playground is unavailable for the same reason. |58| **Stored [image](/developers/model-capabilities/images/generation) and [video](/developers/model-capabilities/video/generation) outputs** | Image and video generation can have SpaceXAI host the generated media and return it as a `file_id` or URL. Under ZDR there is no SpaceXAI-side output storage: no `file_id` inputs and no stored outputs. For images, return base64 or supply your own pre-signed upload URLs in `output.upload_urls` (one per image, required for deferred image requests). For video, you must supply your own `output.upload_url`. In both cases SpaceXAI uploads the result to your URL via HTTP `PUT` and stores nothing itself. Image generation is also limited to `grok-imagine` models and excludes agentic image generation. The Console video playground asks for a pre-signed upload URL before each video it generates, plus an optional pre-signed view URL to watch the video. |