1#### Model Capabilities
2
3# Best Practices Guide
4
5This guide collects the practices that get the best results from Grok text and reasoning models on the xAI API. It covers which API to use, how to carry reasoning across turns, how to lower cost with prompt caching, how to configure requests, and how to run evaluations.
6
7Start with these settings:
8
9| Setting | Recommendation |
10|---|---|
11| [API](#use-the-responses-api) | Use the [Responses API](/developers/model-capabilities/text/generate-text). Chat Completions is deprecated. |
12| [Multi-turn history](#send-encrypted-reasoning-back-with-every-turn) | Send the encrypted reasoning content back to the API with every turn. |
13| [Prompt caching](#use-prompt-caching-to-lower-cost) | Give each conversation its own `prompt_cache_key`, and only append to the conversation. |
14| [Reasoning effort](#reasoning-effort) | Set it explicitly on every request, and record it with every result. |
15| [Streaming and timeouts](#streaming-and-timeouts) | Prefer streaming. Allow at least 900 seconds per request, and 1800 seconds or more for long-running evals. |
16| [Output length](#output-length) | Set `max_output_tokens` above your longest expected answer, or leave it at its 128,000-token default. |
17| [Sampling](#sampling-parameters) | Set `temperature` to 0.7 and `top_p` to 0.95, and don't stray far from them. |
18
19## Use the Responses API
20
21Use the Responses API (`POST /v1/responses`) with Grok reasoning models. The Chat Completions API is deprecated, and it has no field for the model's encrypted reasoning, so every multi-turn or tool-calling conversation sent through it discards the model's earlier thinking. New capabilities also arrive on the Responses API first.
22
23If your code already uses Chat Completions, the [comparison guide](/developers/model-capabilities/text/comparison) maps each parameter and response field to its Responses API equivalent.
24
25> [!WARNING]
26>
27> Chat Completions is offered only as a legacy endpoint. Results from it understate what Grok reasoning models can do, especially across multiple turns and tool calls.
28
29## Send encrypted reasoning back with every turn
30
31Grok reasoning models think before they answer. The Responses API returns that reasoning as an encrypted `reasoning` item in the response's `output` array, alongside the message and any function calls. When you send these items back unchanged in the next request's `input`, the model stays aware of its earlier reasoning traces. This drastically improves performance on multi-turn conversations and agent loops, where each step builds on decisions the model already reasoned through.
32
33There are two ways to keep reasoning in the conversation:
34
35* Chain requests with `previous_response_id` while `store` is on (the default). SpaceXAI stores the reasoning with the response and restores it for you.
36* Manage the history yourself with `store: false`. Request the reasoning with `include: ["reasoning.encrypted_content"]` and append every output item to the next request's `input`. Each request is then self-contained and carries the full conversation, which you can log and replay.
37
38`grok-4.7` returns `reasoning.encrypted_content` on every Responses API response, whether or not `include` lists it. Your code still has to send it back. See [Adding encrypted thinking content](/developers/model-capabilities/text/generate-text#adding-encrypted-thinking-content) for the full reference.
39
40The following loop runs a scripted multi-turn conversation. It streams each turn, carries the encrypted reasoning forward, reuses one `prompt_cache_key` for the whole conversation, and prints how many input tokens were served from the cache:
41
42```python customLanguage="pythonOpenAISDK"
43import os
44import uuid
45
46import httpx
47from openai import OpenAI
48
49client = OpenAI(
50 api_key=os.getenv("XAI_API_KEY"),
51 base_url="https://api.x.ai/v1",
52 timeout=httpx.Timeout(900.0),
53)
54
55conversation_id = str(uuid.uuid4())
56history = [{"role": "system", "content": "You are a careful analyst. Answer concisely."}]
57
58turns = [
59 "A train leaves at 9:40 and arrives at 13:05. How long is the trip?",
60 "The return trip is 20% faster. When does it arrive if it leaves at 15:00?",
61]
62
63for turn in turns:
64 history.append({"role": "user", "content": turn})
65 stream = client.responses.create(
66 model="grok-4.7",
67 input=history,
68 reasoning={"effort": "high"},
69 temperature=0.7,
70 top_p=0.95,
71 include=["reasoning.encrypted_content"],
72 store=False,
73 stream=True,
74 extra_body={"prompt_cache_key": conversation_id},
75 )
76 response = None
77 for event in stream:
78 if event.type == "response.output_text.delta":
79 print(event.delta, end="", flush=True)
80 elif event.type in ("response.completed", "response.incomplete"):
81 response = event.response
82 print()
83 # Append every output item, including reasoning items with encrypted_content.
84 history.extend(response.output)
85 print(f"Cached tokens: {response.usage.input_tokens_details.cached_tokens}")
86```
87
88```javascript customLanguage="javascriptOpenAISDK"
89import { randomUUID } from "node:crypto";
90import OpenAI from "openai";
91
92const client = new OpenAI({
93 apiKey: process.env.XAI_API_KEY,
94 baseURL: "https://api.x.ai/v1",
95 timeout: 900000,
96});
97
98const conversationId = randomUUID();
99const history = [
100 { role: "system", content: "You are a careful analyst. Answer concisely." },
101];
102
103const turns = [
104 "A train leaves at 9:40 and arrives at 13:05. How long is the trip?",
105 "The return trip is 20% faster. When does it arrive if it leaves at 15:00?",
106];
107
108for (const turn of turns) {
109 history.push({ role: "user", content: turn });
110 const stream = await client.responses.create({
111 model: "grok-4.7",
112 input: history,
113 reasoning: { effort: "high" },
114 temperature: 0.7,
115 top_p: 0.95,
116 include: ["reasoning.encrypted_content"],
117 store: false,
118 stream: true,
119 prompt_cache_key: conversationId,
120 });
121 let response;
122 for await (const event of stream) {
123 if (event.type === "response.output_text.delta") {
124 process.stdout.write(event.delta);
125 } else if (event.type === "response.completed" || event.type === "response.incomplete") {
126 response = event.response;
127 }
128 }
129 console.log();
130 // Append every output item, including reasoning items with encrypted_content.
131 history.push(...response.output);
132 console.log(`Cached tokens: ${response.usage.input_tokens_details.cached_tokens}`);
133}
134```
135
136```python customLanguage="pythonXAI"
137import os
138import uuid
139
140from xai_sdk import Client
141from xai_sdk.chat import system, user
142
143conversation_id = str(uuid.uuid4())
144
145client = Client(
146 api_key=os.getenv("XAI_API_KEY"),
147 timeout=900,
148 # Routes every turn of this conversation to the same server for cache reuse.
149 metadata=(("x-grok-conv-id", conversation_id),),
150)
151
152chat = client.chat.create(
153 model="grok-4.7",
154 reasoning_effort="high",
155 temperature=0.7,
156 top_p=0.95,
157 store_messages=False,
158 use_encrypted_content=True,
159)
160chat.append(system("You are a careful analyst. Answer concisely."))
161
162turns = [
163 "A train leaves at 9:40 and arrives at 13:05. How long is the trip?",
164 "The return trip is 20% faster. When does it arrive if it leaves at 15:00?",
165]
166
167for turn in turns:
168 chat.append(user(turn))
169 for response, chunk in chat.stream():
170 print(chunk.content, end="", flush=True)
171 print()
172 # Appending the response also appends its encrypted reasoning.
173 chat.append(response)
174 print(f"Cached tokens: {response.usage.cached_prompt_text_tokens}")
175```
176
177The Vercel AI SDK carries encrypted reasoning automatically when you chain requests with `previousResponseId`; see [Reasoning](/developers/model-capabilities/text/reasoning#encrypted-reasoning-content).
178
179### Keep reasoning in agent loops
180
181Tool-calling workloads need the same treatment. When the model calls a function, its reasoning item arrives in the same `output` array as the `function_call`. Append the whole array, then your `function_call_output` items, so the model resumes with its plan intact when it reads the tool result:
182
183```python customLanguage="pythonOpenAISDK"
184import json
185
186# client, tools, task_prompt, run_tool, and conversation_id come from your code.
187def stream_turn(input_items):
188 stream = client.responses.create(
189 model="grok-4.7",
190 input=input_items,
191 tools=tools,
192 reasoning={"effort": "high"},
193 temperature=0.7,
194 top_p=0.95,
195 include=["reasoning.encrypted_content"],
196 store=False,
197 stream=True,
198 extra_body={"prompt_cache_key": conversation_id},
199 )
200 for event in stream:
201 if event.type in ("response.completed", "response.incomplete"):
202 return event.response
203
204input_items = [{"role": "user", "content": task_prompt}]
205
206while True:
207 response = stream_turn(input_items)
208 # Reasoning, messages, and function calls all go back in order.
209 input_items.extend(response.output)
210
211 calls = [item for item in response.output if item.type == "function_call"]
212 if not calls:
213 break
214
215 for call in calls:
216 result = run_tool(call.name, json.loads(call.arguments))
217 input_items.append({
218 "type": "function_call_output",
219 "call_id": call.call_id,
220 "output": json.dumps(result),
221 })
222
223print(response.output_text)
224```
225
226Code drops reasoning most often in one of these ways:
227
228* It extracts `output_text` and appends it as a plain assistant message, discarding the reasoning and function call items around it.
229* It edits, reorders, or filters output items before sending them back. Pass them back exactly as received.
230* It rebuilds the conversation from a transcript in a different format, such as a Chat Completions `messages` array, between turns.
231
232For long agent loops, [WebSocket mode](/developers/advanced-api-usage/websocket-mode) reduces per-turn latency, and [context compaction](/developers/advanced-api-usage/context-compaction) keeps long conversations within the context window.
233
234## Use prompt caching to lower cost
235
236The xAI API caches prompts automatically, and cached input tokens are billed at a lower rate than regular input tokens; see [Pricing](/developers/pricing) for each model's cached token price. The cache matches your input from the start: when a request begins with exactly the same items as an earlier request, that matching prefix is served from the cache, and only the new items are processed. Multi-turn conversations and agent loops benefit the most, because each turn resends everything before it.
237
238To get the most cache hits:
239
240* Give each conversation its own [`prompt_cache_key`](/developers/advanced-api-usage/prompt-caching/maximizing-cache-hits) and reuse it on every turn. Cache entries live on individual servers, and the key routes every turn of the conversation to the same one. A UUID or your application's session ID works well. With the xAI SDK, send the same value as `x-grok-conv-id` gRPC metadata.
241* Only append to the conversation. Editing, removing, or reordering earlier items breaks the prefix, and everything after the change is processed at the full price.
242* Send the encrypted reasoning back unchanged. For reasoning models, leaving it out is the most common cause of cache misses. Chaining with `previous_response_id` keeps the prefix intact as well.
243* Put static content first. System prompts, few-shot examples, and reference documents at the start of the input form a stable prefix that every turn can reuse.
244
245Check `usage.input_tokens_details.cached_tokens` on each response. It should grow with each turn of a conversation; if it stays at 0, check that every turn sends the same `prompt_cache_key` and that earlier items are unchanged. Caching works with streaming and with tool calls, but cache hits are not guaranteed, because entries can be evicted under load. See [What Breaks Caching](/developers/advanced-api-usage/prompt-caching/multi-turn) for examples.
246
247## Configure requests for reasoning models
248
249Reasoning models spend tokens and time thinking before they respond, so reasoning effort, streaming and timeouts, output limits, and sampling parameters all affect your results.
250
251### Reasoning effort
252
253On the Responses API, `reasoning.effort` controls how much the model thinks. The supported values and the default depend on the model; `grok-4.7` defaults to `"high"`, while `grok-4.3` defaults to `"low"` and also accepts `"none"`. Set the effort explicitly on every request so a default never decides your result, and choose the effort you intend to run in production. If cost or latency matters, compare effort levels and weigh quality against tokens and time. See the [`reasoning_effort`](/developers/model-capabilities/text/reasoning#the-reasoning_effort-parameter) parameter for the supported values per model.
254
255### Streaming and timeouts
256
257Prefer [streaming](/developers/model-capabilities/text/streaming) (`stream: true`) to non-streaming requests. A streamed response sends events as the model works, which keeps the connection from dropping during long generations, and the final `response.completed` event carries the full response, including the encrypted reasoning items to send back.
258
259Set the client timeout to at least 900 seconds, as the examples in this guide do. High reasoning effort on complex problems can run for several minutes, and long-running evals such as Humanity's Last Exam (HLE) can take upwards of 1800 seconds per request. Raise the timeout above your longest expected request so slow requests are not recorded as failures.
260
261### Output length
262
263`max_output_tokens` caps only the visible answer; reasoning and function call tokens do not count toward it. When unset, it defaults to 128,000 tokens. A tight cap cuts the answer off, so set the cap above the longest answer you expect, and treat responses with `status: "incomplete"` as truncated.
264
265### Sampling parameters
266
267> [!WARNING]
268>
269> Set `temperature` to 0.7 and `top_p` to 0.95, and don't deviate far from these values. Grok models don't respond well to other sampling settings, and quality usually drops when you change them.
270
271Set both values explicitly on every request so your results don't depend on server defaults, and use the same values for every model you compare.
272
273### Errors, retries, and rate limits
274
275Separate infrastructure failures from model failures. Retry `429` and `5xx` responses with exponential backoff, and rerun any request that still fails until it gets a response. Large workloads should respect your team's [rate limits](/developers/rate-limits).
276
277## Evaluate Grok models
278
279A good evaluation tells you how a model will behave on your workload, which is rarely the same as how it scores on a public leaderboard. Apply the settings above, then design and run the eval with these practices.
280
281### Design the eval
282
283Start by writing down what success means for your application in terms you can measure. "Answers support questions well" is too vague to test. A testable version is "resolves at least 85% of a 500-ticket sample without inventing a policy, with a median latency under 20 seconds." Most applications need several criteria at once, such as task accuracy, instruction following, format compliance, safety, latency, and cost.
284
285Build the dataset from the work the model will actually do. Production logs, with personal data removed, are the most representative source; add hand-written edge cases, adversarial inputs, and long or multi-turn examples. Check whether public benchmark items might appear in training data, and keep in mind each model's knowledge cutoff (May 2026 for `grok-4.7`) when questions depend on recent events. If the task needs current information, enable [web search](/developers/tools/web-search) and evaluate that configuration directly.
286
287Choose the cheapest grader that is reliable for the criterion:
288
289| Grader | Best for | Watch out for |
290|---|---|---|
291| Code | Exact answers, unit tests, schema validation, tool-call arguments | Rejecting correct answers in an unexpected format |
292| Model as judge | Open-ended quality, rubric scoring, pairwise comparison | Position bias, preference for longer answers, drift from human judgment |
293| Human review | Calibrating other graders, high-stakes or subjective tasks | Cost, speed, and disagreement between reviewers |
294
295When you use a model as a judge, give it a specific rubric with examples of each score, ask it to reason before it scores, and randomize the order of answers in pairwise comparisons. Measure its agreement with human labels on a sample before you trust it at scale.
296
297Prompt each model fairly. Grok reasoning models reason before they answer, so state the task, constraints, and output format directly. If you tuned a prompt for another model, spend comparable effort tuning it for Grok before you compare scores.
298
299### Run, measure, and compare
300
301Reasoning models are not deterministic, so a single run can mislead you. Run each item several times, report the mean and spread, and use pass@k where any one correct attempt counts. Treat small differences between models with suspicion until they hold across runs.
302
303If you must exclude an item that keeps failing, exclude it for every model you compare so all of them are scored on the same set, and report how many items you excluded; dropping failures for one model alone inflates its score.
304
305Log the full response for every request, including its `usage` object. `usage.output_tokens_details.reasoning_tokens` reports how many tokens the model spent reasoning, `usage.input_tokens_details.cached_tokens` shows how much of the input came from the cache, and `usage.cost_in_usd_ticks` gives the exact billed cost of each request; together with wall-clock latency, they let you compare quality per dollar and per second. See [Cost Tracking](/developers/cost-tracking) for converting ticks to dollars. Record the model name, reasoning effort, sampling parameters, date, and harness version with every result, because models and harnesses change.
306
307Before you report results, confirm that:
308
309* Every request used the Responses API.
310* Multi-turn and tool-calling items carried their encrypted reasoning forward unchanged.
311* Reasoning effort, sampling parameters, model name, and harness version are recorded and match across the models you compare.
312* Failed requests were retried, any exclusions apply to every model equally, and truncated responses were flagged.
313
314## Related
315
316* [Generate Text](/developers/model-capabilities/text/generate-text): the Responses API, stored conversations, and encrypted thinking content
317* [Reasoning](/developers/model-capabilities/text/reasoning): reasoning effort, encrypted reasoning, and reasoning summaries
318* [Streaming](/developers/model-capabilities/text/streaming): streaming responses with Server-Sent Events
319* [Prompt Caching](/developers/advanced-api-usage/prompt-caching): how caching works and how to maximize cache hits
320* [Function Calling](/developers/tools/function-calling): defining tools and handling tool calls
321* [Comparison with Chat Completions](/developers/model-capabilities/text/comparison): migrating from Chat Completions
