2 2
3> For the complete documentation index, see [llms.txt](/llms.txt). Markdown versions of documentation pages are available by appending `.md` to the page URL.3> For the complete documentation index, see [llms.txt](/llms.txt). Markdown versions of documentation pages are available by appending `.md` to the page URL.
4 4
5## Prompt caching fundamentals
6
5Model prompts often contain repetitive content, like system prompts and common instructions. OpenAI routes API requests to servers that recently processed the same prompt, making it faster and less expensive to reuse an exact prompt prefix than to process it from scratch. Prompt Caching works automatically for eligible requests, with no code changes required. It is enabled for all recent [models](https://developers.openai.com/api/docs/models), `gpt-4o` and newer.7Model prompts often contain repetitive content, like system prompts and common instructions. OpenAI routes API requests to servers that recently processed the same prompt, making it faster and less expensive to reuse an exact prompt prefix than to process it from scratch. Prompt Caching works automatically for eligible requests, with no code changes required. It is enabled for all recent [models](https://developers.openai.com/api/docs/models), `gpt-4o` and newer.
6 8
7Cache writes have no additional fee on models before the GPT-5.6 family. For GPT-5.6 models and later model families, cache writes cost 1.25× the uncached input token rate. On these models, both implicit and explicit caching are more consistent and reliable. You can also use explicit cache breakpoints to control exactly which prompt prefixes OpenAI caches. OpenAI reports writes in `cache_write_tokens` and reads in `cached_tokens`, so you can measure the cost of writes against the savings from later cache hits.9This guide describes how prompt caching works in detail, so that you can optimize your prompts for lower latency and cost.
8 10
9This guide describes how Prompt Caching works in detail, so that you can optimize your prompts for lower latency and cost.11### Caching best practices
10 12
11## Caching behavior changes when migrating to GPT-5.613Cache hits are only possible for exact prefix matches within a prompt. To realize caching benefits, place static content like instructions and examples at the beginning of your prompt, and put variable content, such as user-specific information, at the end. This also applies to images and tools, which must be identical between requests.
12 14
13GPT-5.6 models and later model families cache exact prompt prefixes at cache15- Keep instructions, tools, schemas, and shared context stable. Place request-specific content after the reusable prefix.
14breakpoints. By default, the service places an implicit breakpoint at the latest16- Set [`prompt_cache_key`](https://developers.openai.com/api/reference/resources/responses/methods/create#responses-create-prompt_cache_key) on requests that share long, common prompt prefixes. Reuse the same key for those requests to help improve cache hit rates.
15user or tool message. Unlike earlier models, it does not automatically fall17- Monitor cache reads with `cached_tokens`. On GPT-5.6 and later, use `cache_write_tokens` to compare cache-write costs with later cache reads.
16back to the longest matching unmarked prefix before that breakpoint.
17 18
18For example, requests might share 4,000 tokens of instructions and other static19
19content, followed by changing timestamps, tool-call history, or user input. If
20the implicit breakpoint includes that changing content, the full prefix at the
21breakpoint differs between requests. As a result, `cached_tokens` can be `0`
22even though the requests share thousands of identical tokens, and the service
23can repeatedly write the changing prefix to cache.
24 20
25To reuse the shared content, add an explicit `prompt_cache_breakpoint` at the21### How prompt caching works
26end of the stable prefix and set the same `prompt_cache_key` on requests that
27share it. Content after the breakpoint can then change without invalidating the
28cached prefix.
29 22
30To avoid cache-write charges for the changing suffix, set23By default, caching is enabled automatically for prompts that are 1,024 tokens or longer. When you make an API request, the following steps occur:
31`prompt_cache_options.mode` to `explicit`. This disables the implicit
32breakpoint, so only your explicit breakpoints are eligible for cache reads and
33writes. On GPT-5.6 models and later model families, cache writes cost 1.25× the
34uncached input token rate, so caching only the reusable prefix can reduce costs.
35 24
36See [Prompt cache breakpoints](#prompt-cache-breakpoints) for request examples,251. **Cache routing**
37supported content blocks, and cache policy options.
38 26
39## Structuring prompts27 Requests are routed to a machine based on `prompt_cache_key`, with a hash of the initial prefix of the prompt as a secondary key.
40 28
41Cache hits are only possible for exact prefix matches within a prompt. To realize caching benefits, place static content like instructions and examples at the beginning of your prompt, and put variable content, such as user-specific information, at the end. This also applies to images and tools, which must be identical between requests.292. **Cache lookup**
42 30
4331 The system checks whether the initial portion (prefix) of your prompt exists in the cache on the selected machine.
44 32
45## How it works333. **Cache hit**
46 34
47By default, caching is enabled automatically for prompts that are 1024 tokens or longer. When you make an API request, the following steps occur:35 If a matching prefix is found, the system uses the cached result. This decreases latency and bills those tokens at the cached-input rate.
48 36
491. **Cache Routing**:374. **Cache miss**
50 38
51- Requests are routed to a machine based on a hash of the initial prefix of the prompt. The hash typically uses the first 256 tokens, though the exact length varies depending on the model.39 If no matching prefix is found, the system processes your full prompt. When automatic caching is enabled, it may write an eligible prefix to cache on that machine for future requests.
52- If you provide the [`prompt_cache_key`](https://developers.openai.com/api/reference/resources/responses/methods/create#responses-create-prompt_cache_key) parameter, it is combined with the prefix hash, allowing you to influence routing and improve cache hit rates. This is especially beneficial when many requests share long, common prefixes.
53 40
542. **Cache Lookup**: The system checks if the initial portion (prefix) of your prompt exists in the cache on the selected machine.41For GPT-5.6 and later, 1,024 tokens is a strict minimum. For earlier models,
553. **Cache Hit**: If a matching prefix is found, the system uses the cached result. This decreases latency and bills those tokens at the cached-input rate.42 the minimum varies by model from 1,024 to 2,048 tokens, so prompts just above
564. **Cache Miss**: If no matching prefix is found, the system processes your full prompt. When automatic caching is enabled, it may cache an eligible prefix on that machine for future requests. On GPT-5.6 models and later model families, tokens written to cache are billed at the cache-write rate.43 1,024 tokens may not cache consistently.
57 44
58### Improve cache hit rates with a prompt cache key45### How caching differs by model
59 46
60Set `prompt_cache_key` on requests that share long, common prompt prefixes. Reuse the same key for those requests to help route them to the same cache and improve cache hit rates.47| Behavior | GPT-5.6 and later | Earlier models |
48| -------------------------- | ------------------------------------------------------- | ------------------------------------------------------------------- |
49| Cache matching | Exact matching at eligible cache breakpoints | Automatic best-effort reuse of matching prefixes |
50| Explicit cache breakpoints | Supported. Implicit caching is also available. | Not supported. Caching is automatic. |
51| Minimum cacheable prefix | 1,024 tokens | 1,024 to 2,048 tokens, depending on the model |
52| Cache write charges | 1.25× the uncached input token rate | No additional cache-write fee |
53| Cache lifetime | 30-minute exact TTL set with `prompt_cache_options.ttl` | Model-dependent maximum retention set with `prompt_cache_retention` |
61 54
62On GPT-5.6 models and later model families, you must set `prompt_cache_key` to use the more reliable matching for both implicit and explicit caching. At each breakpoint, the service matches the key with the exact prompt prefix. Without a key, requests may still receive automatic cache hits, but they do not use the improved matching.55For GPT-5.6 and later models, see [Prompt caching for GPT-5.6 and later models](#prompt-caching-for-gpt-56-and-later-models). For earlier models, see [Prompt caching for earlier models](#prompt-caching-for-earlier-models).
63 56
64Keep the total traffic across all prefixes for each key to approximately 15 requests per minute. If a key receives a higher rate, some requests may miss the cache. For higher-volume workloads, partition traffic across more keys and use a stable mapping so requests with the same key continue to share prefixes.57## Prompt caching for GPT-5.6 and later models
58
59GPT-5.6 and later model families cache exact prompt prefixes at cache breakpoints. By default, the service places an implicit breakpoint at the latest user or tool message. Unlike earlier models, it does not automatically fall back to the longest matching unmarked prefix before that breakpoint.
60
61To improve cache reuse, identify the prompt content that stays the same across requests. Then choose a breakpoint that ends after that content and use a consistent `prompt_cache_key`.
62
63### How cache breakpoints work
64
65A cache breakpoint marks the end of a reusable prompt prefix. The prefix includes the marked content block and all prompt content rendered before it. Content after the breakpoint can change without invalidating that prefix.
66
67For a prefix to be eligible for caching, it must contain at least 1,024 tokens through the breakpoint. The minimum applies to the complete rendered prefix, not just the marked content block.
68
69**Cache writes and cache reads**
70
71A cache write creates an entry for an eligible prompt prefix. A cache read reuses an entry that an earlier request wrote.
72
731. The first request writes an eligible prefix at a cache breakpoint.
742. A later request can read that prefix when the content through an eligible breakpoint matches the earlier cache entry and the two requests share the same `prompt_cache_key`.
753. A change before the breakpoint changes the prefix and will prevent a cache hit.
764. A change after the breakpoint does not invalidate the earlier cached prefix.
77
78Repeated prompt content alone does not guarantee a cache hit. If no matching entry was written at an eligible breakpoint, the system cannot read that prefix from the cache.
79
80**When the default breakpoint works**
81
82Implicit caching works well when a conversation grows by appending new messages and the earlier conversation history stays the same.
83
84```text
85Request 1: Instructions → User message 1 [implicit breakpoint]
86Request 2: Instructions → User message 1 → Assistant message 1 → User message 2 [implicit breakpoint]
87```
88
89The first request can write the prefix through user message 1. On the next request, that earlier breakpoint can provide a cache read. The newly appended content can then be written at the latest implicit breakpoint.
90
91**When changing content prevents reuse**
92
93Some applications send separate requests that share the same instructions but have different timestamps and user messages. Unlike successive turns in a conversation, these requests do not share conversation history.
94
95```text
96Request 1: Stable instructions → Timestamp 1 → User message 1 [implicit breakpoint]
97Request 2: Stable instructions → Timestamp 2 → User message 2 [implicit breakpoint]
98```
99
100The first request writes a prefix that includes timestamp 1 and user message 1. On the second request, timestamp 2 and user message 2 change the prefix at the breakpoint. If no earlier matching entry exists, `cached_tokens` can be `0` and the service can write the changing prefix again.
101
102Add an explicit breakpoint at the end of the stable content to make that content reusable:
103
104```text
105Stable instructions [explicit breakpoint] → Timestamp → User message
106```
107
108The first request writes the stable prefix. Later requests with the same prefix and `prompt_cache_key` can read that entry, even when the timestamp and user message change.
109
110### Choose a caching mode
65 111
66## Prompt cache breakpoints112Use `prompt_cache_options.mode` to set the request-wide caching policy.
67 113
68For GPT-5.6 models and later model families, you can mark the end of a reusable prompt prefix with an explicit cache breakpoint. Breakpoints are available in both the Responses API and Chat Completions API.114**Implicit caching**
69 115
70Set the request-wide cache policy with `prompt_cache_options.mode`:116- `implicit` is the default. OpenAI places a cache breakpoint on the latest user or tool message and also uses any explicit breakpoints you provide.
117- Use implicit caching when the prompt grows by appending reusable content. Earlier eligible breakpoints can provide cache reads, while the latest message creates a new checkpoint for future requests.
71 118
72- `implicit` is the default. OpenAI places a cache breakpoint on the latest message and also uses any explicit breakpoints you provide.119**Explicit breakpoints with implicit caching**
73- `explicit` disables the implicit breakpoint. Only explicit breakpoints are used for cache reads and writes. If the conversation contains no explicit breakpoints, the request does not use prompt caching or incur cache-write charges.
74 120
75Add `prompt_cache_breakpoint: { "mode": "explicit" }` to a supported prompt content block. The breakpoint marks the exact end of the cached prefix, including that block and all prompt content rendered before it. Content after the breakpoint can change without invalidating the earlier cached prefix. All breakpoints use the request-wide `prompt_cache_options.ttl`, which currently defaults to `30m` and is the only supported value.121- You can add an explicit breakpoint without changing the default caching mode. This lets requests read a stable prefix while the implicit breakpoint continues to cache the latest eligible message.
122- This approach is useful when both the shared prefix and the growing conversation history are likely to be reused. However, the latest implicit breakpoint can still write a changing suffix to the cache.
76 123
77Each request can create up to four new cache writes. Breakpoints from earlier conversation turns are read-only: they can match the cache, but the request does not write them again. In `implicit` mode, the breakpoint on the latest message uses one write slot, so up to the latest three explicit breakpoints can be written. In `explicit` mode, up to the latest four explicit breakpoints can be written. For cache reads, OpenAI considers up to the latest 50 breakpoints in the conversation.124**Explicit-only caching**
78 125
79Responses API supports breakpoints on `input_text`, `input_image`, and `input_file` blocks. Chat Completions API supports them on `text`, `image_url`, `input_audio`, `file`, and `refusal` blocks.126- Set `prompt_cache_options.mode` to `explicit` to disable the implicit breakpoint. Only explicit breakpoints are used for cache reads and writes.
127- Use explicit-only mode when the prompt has a stable prefix followed by request-specific content that is unlikely to be reused. This caches the reusable prefix without creating a new cache write for the changing suffix.
128- Adding an explicit breakpoint does not automatically switch a request to explicit-only mode. If you set `mode` to `explicit` but provide no explicit breakpoints, the request does not use prompt caching or incur cache-write charges.
80 129
81When several breakpoints match cached content, the service reads from the longest matching prefix.130### Add explicit cache breakpoints
82 131
83The following examples are abbreviated to show the request shape. In a real request, the rendered prefix before the marked breakpoint must contain at least 1,024 tokens to be cacheable.132Add `prompt_cache_breakpoint: { "mode": "explicit" }` to the last supported content block in the reusable prefix. The breakpoint includes that block and all prompt content rendered before it.
133
134The following examples are abbreviated to show the request shape. In a real request, the rendered prefix through the marked breakpoint must contain at least 1,024 tokens.
84 135
85 136
86 137
87Responses API138Responses API
88 139
89 140
90 This request uses the default `implicit` mode, which places a breakpoint on141 This request places an explicit breakpoint after stable developer instructions. Explicit-only mode prevents the changing user message from creating an additional implicit cache write.
91 the latest message, and adds an explicit breakpoint after a stable file.
92 142
93```json143```json
94{144{
95 "model": "gpt-5.6",145 "model": "gpt-5.6",
96 "prompt_cache_key": "tenant:acme:knowledge-base-v1",146 "prompt_cache_key": "support:knowledge-base-v1",
147 "prompt_cache_options": {
148 "mode": "explicit"
149 },
97 "input": [150 "input": [
98 {151 {
99 "type": "message",152 "type": "message",
100 "role": "user",153 "role": "developer",
101 "content": [154 "content": [
102 {155 {
103 "type": "input_file",156 "type": "input_text",
104 "file_id": "file_123",157 "text": "Follow the shared support policies and reference material...",
105 "prompt_cache_breakpoint": {158 "prompt_cache_breakpoint": {
106 "mode": "explicit"159 "mode": "explicit"
107 }160 }
161 }
162 ]
108 },163 },
164 {
165 "type": "message",
166 "role": "user",
167 "content": [
109 {168 {
110 "type": "input_text",169 "type": "input_text",
111 "text": "Answer the current question."170 "text": "Where is order 1234?"
112 }171 }
113 ]172 ]
114 }173 }
117```176```
118 177
119 178
179 Top-level `instructions` cannot contain a `prompt_cache_breakpoint`. To mark reusable developer instructions, place them in an `input_text` block inside a developer message, as shown above.
180
120 181
121 182
122 183
125Chat Completions API186Chat Completions API
126 187
127 188
128 This request disables automatic breakpoint placement. Only the marked189 This request marks the system-message prefix. Explicit-only mode limits cache reads and writes to the marked stable content.
129 system-message prefix is eligible for billable cache writes and discounted
130 cache reads.
131 190
132```json191```json
133{192{
134 "model": "gpt-5.6",193 "model": "gpt-5.6",
135 "prompt_cache_key": "tenant:acme:support-assistant-v1",194 "prompt_cache_key": "support:knowledge-base-v1",
136 "prompt_cache_options": {195 "prompt_cache_options": {
137 "mode": "explicit"196 "mode": "explicit"
138 },197 },
142 "content": [201 "content": [
143 {202 {
144 "type": "text",203 "type": "text",
145 "text": "You are a support assistant.",204 "text": "You are a support assistant. Follow the shared policies...",
146 "prompt_cache_breakpoint": {205 "prompt_cache_breakpoint": {
147 "mode": "explicit"206 "mode": "explicit"
148 }207 }
159 218
160 219
161 220
162Only `explicit` is valid for `prompt_cache_breakpoint.mode`. A marker on an unsupported or non-cacheable block returns a `400 invalid_request_error`. Older models also reject `prompt_cache_options` and `prompt_cache_breakpoint`; continue using their existing automatic prompt caching behavior.221To combine an explicit breakpoint with the default implicit breakpoint, omit `prompt_cache_options.mode` or set it to `implicit`.
222
223**Supported content blocks**
224
225The Responses API supports breakpoints on `input_text`, `input_image`, and `input_file` blocks. The Chat Completions API supports them on `text`, `image_url`, `input_audio`, `file`, and `refusal` blocks.
226
227Only `explicit` is valid for `prompt_cache_breakpoint.mode`. A marker on an unsupported or non-cacheable block returns a `400 invalid_request_error`.
228
229Tool definitions, structured output schemas, messages, images, and files can contribute to the rendered prefix. Keep the content, order, and relevant settings identical across requests that should share a cache.
230
231### Use multiple cache breakpoints
232
233Use multiple explicit breakpoints when parts of a prompt change at different rates. For example, shared instructions can stay stable while reference material is updated more often. Separate breakpoints let requests reuse the longest eligible prefix that remains unchanged.
234
235Each request can create up to four new cache writes. Breakpoints from earlier conversation turns are read-only: they can match the cache, but the request does not write them again. If more than four breakpoints are set, only the last four are written.
236
237In `implicit` mode, the breakpoint on the latest message uses one write slot. Up to the latest three explicit breakpoints can use the remaining slots. In `explicit` mode, up to the latest four explicit breakpoints can create new cache writes.
238
239For cache reads, OpenAI considers up to the latest 50 breakpoints in the conversation. When several breakpoints match cached content, the service reads from the longest matching prefix.
240
241### Improve cache matching with a prompt cache key
242
243Set `prompt_cache_key` on requests that share long, common prompt prefixes. Reuse the same key for those requests to help route them to the same cache and improve cache hit rates. Common values for `prompt_cache_key` include session IDs and user IDs.
244
245For GPT-5.6, you must set `prompt_cache_key` to use the more reliable matching for both implicit and explicit caching. At each breakpoint, the service matches the key with the exact prompt prefix. Without a key, requests may still receive automatic cache hits, but they do not use the improved matching.
246
247Keep the total traffic across all prefixes for each key to approximately 15 requests per minute. If a key receives a higher rate, some requests may miss the cache. For higher-volume workloads, partition traffic across more keys and use a stable mapping so requests with the same key continue to share prefixes.
248
249### Measure cache reads and writes
250
251Monitor `cached_tokens` and `cache_write_tokens` to understand whether your breakpoint placement produces cache reuse or repeated cache writes.
252
253`cached_tokens` is the number of input tokens read from the cache. `cache_write_tokens` is the number of input tokens newly written to the cache.
254
255For the Responses API, both fields appear in `usage.input_tokens_details`. For the Chat Completions API, they appear in `usage.prompt_tokens_details`.
256
257```json
258{
259 "usage": {
260 "input_tokens": 2600,
261 "input_tokens_details": {
262 "cached_tokens": 2000,
263 "cache_write_tokens": 400
264 }
265 }
266}
267```
268
269In this example, 2,000 tokens were read from the cache and 400 additional tokens were written. The remaining 200 input tokens were neither read nor written. A longer cache write does not bill the already cached 2,000 tokens again.
270
271**Understand cache-write pricing**
272
273Cache reads, cache writes, and ordinary input tokens are separate billing categories.
163 274
164## Prompt cache retention2751. Cached input tokens are billed at 0.1× the uncached input token rate.
2762. Tokens written to the cache are billed at 1.25× the uncached input token rate.
2773. Tokens that are neither read nor written are billed at the uncached input token rate.
165 278
166Prompt caching has two controls with different semantics:279The 1.25× cache-write rate is the total rate for written tokens. It is not an additional charge on top of another full input-token charge. A breakpoint does not create a charge by itself. Charges apply to tokens that are actually written to the cache.
167 280
168- For GPT-5.6 models and later model families, `prompt_cache_options.ttl` sets a minimum cache lifetime. It does not select a storage policy or maximum retention period.281Repeated writes increase cost when the resulting cache entries are not reused. If `cache_write_tokens` stays high while `cached_tokens` remains low, check whether an implicit breakpoint includes content that changes between requests.
169- For earlier models, `prompt_cache_retention` selects a maximum-retention policy. This field is deprecated for GPT-5.6 models and later model families.
170 282
171For GPT-5.6 models and later model families, use `prompt_cache_options.ttl` to set the minimum lifetime of all breakpoints written by the request. The only supported value is `30m`, which is also the default. A cached prefix remains eligible for reuse for at least 30 minutes, but OpenAI may retain it longer.283### Set cache lifetime
172 284
173For models before the GPT-5.6 family, continue to set `prompt_cache_retention` on your `Responses.create` request or `chat.completions.create` request. For models that support both in-memory and extended retention, prompt cache pricing is the same for both policies.285Use `prompt_cache_options.ttl` to set the lifetime of all breakpoints written by a request. The only supported value is `30m`, which is also the default.
174 286
175### In-memory prompt cache retention287The 30-minute lifetime begins when the prefix is written and refreshes whenever the prefix is reused. A cached prefix remains eligible for reuse for 30 minutes after its most recent write or reuse, though OpenAI may retain it longer.
288
289Reusing a cached prefix refreshes its lifetime without creating another cache-write charge.
290
291### Troubleshoot common caching issues
292
293- **`cached_tokens` is zero:** Check that the rendered prefix through the breakpoint contains at least 1,024 tokens. Confirm that an earlier request wrote the same prefix and that related requests use the same `prompt_cache_key`.
294
295- **Cache writes repeat on every request:** Check whether a timestamp, changing user input, tool-call history, or other request-specific content appears before the eligible breakpoint. Move the explicit breakpoint to the end of the stable prefix.
296
297- **Cache reads and writes are both nonzero:** In implicit mode, a request can read an earlier cached prefix and write newly appended content at the latest breakpoint. Use explicit-only mode if that new content should not be cached.
298
299- **Explicit mode produces no cache hits:** Confirm that at least one supported content block has `prompt_cache_breakpoint: { "mode": "explicit" }` and that the rendered prefix through the marker meets the 1,024-token minimum.
300
301- **Cache hits decrease at higher request volumes:** Keep traffic for each `prompt_cache_key` to approximately 15 requests per minute. Use stable, deterministic keys to partition larger workloads.
302
303- **A previously cached prompt no longer matches:** Check whether tool definitions, tool ordering, structured output schemas, images, prompt content, or request settings changed before the breakpoint.
304
305- **A breakpoint is rejected:** Attach the marker to a supported content block and use `explicit` as its mode. Do not attach a breakpoint to top-level Responses API `instructions`.
306
307## Prompt caching for earlier models
308
309Earlier models use automatic prompt caching to reuse matching prompt prefixes. When an eligible request is routed to a machine that recently processed the same prefix, the service can reuse the cached result instead of processing that content again.
310
311Prompt caching works automatically for supported models. A consistent `prompt_cache_key`, stable prompt structure, and an appropriate `prompt_cache_retention` setting can improve cache reuse.
312
313### How automatic prompt caching works
314
315Cache hits are only possible for exact prefix matches within a prompt. When a request arrives, the service checks whether an eligible initial portion of the prompt already exists in the cache on the selected machine.
316
317If a matching prefix is available, the service can reuse an eligible matching prefix and report those tokens in `cached_tokens`. If no match is available, the service processes the full prompt and may cache eligible content for future requests.
318
319Cache reuse is best-effort. A cache hit depends on the prompt prefix remaining identical, the cached content still being available, and the request reaching a machine that holds the matching entry.
320
321For example, separate requests can reuse shared instructions and reference material while the user message changes:
322
323```text
324Request 1: Shared instructions → Shared reference material → User message 1
325Request 2: Shared instructions → Shared reference material → User message 2
326```
327
328When the shared prefix is eligible and available, the second request can reuse that content without requiring additional request-specific cache configuration.
329
330**Minimum cacheable prefix**
331
332The minimum cacheable prefix length varies by model and can range from 1,024 to 2,048 tokens. Prompts just above 1,024 tokens may not be cached consistently.
333
334Cache hits occur in increments of 128 tokens. The number of cached tokens can therefore be smaller than the full length of the shared prompt content.
335
336Make sure the repeated portion of the prompt meets the minimum for the model. A request can exceed the minimum overall and still fail to produce a cache hit if its matching prefix is too short.
337
338### Structure prompts for reuse
339
340Cache hits are only possible for exact prefix matches within a prompt. To realize caching benefits, place static content like instructions and examples at the beginning of your prompt, and put variable content, such as user-specific information, at the end. This also applies to images and tools, which must be identical between requests.
341
342Keep system or developer instructions, shared reference material, examples, tool definitions, and structured output schemas stable. Put user input, request identifiers, timestamps, and other changing content after the reusable prefix.
343
344If a dynamic value is needed only for logging or debugging, consider placing it in request metadata instead of inserting it into the prompt.
345
346**Keep tools and schemas identical**
347
348Tool definitions, tool ordering, and structured output schemas contribute to the prompt prefix. Changes to tool descriptions, parameter schemas, schema keys, or ordering can reduce cache reuse.
349
350When you need to restrict which tools are available on a particular request, keep the underlying `tools` array unchanged and use `allowed_tools` where supported.
351
352**Preserve conversation history**
353
354For multi-turn conversations, append new user and assistant messages instead of rewriting earlier messages. Changing, deleting, or reordering earlier content changes the prefix and can cause a cache miss.
355
356Context truncation, summarization, and compaction can reduce prompt size, but they can also reset the reusable prefix. Balance the savings from shorter prompts against the loss of existing cache reuse.
357
358### Improve cache hit rates with a prompt cache key
359
360Set `prompt_cache_key` on requests that share long, common prompt prefixes. Reuse the same key for those requests to help route them to the same cache and improve cache hit rates.
361
362Requests are routed based on the initial prompt prefix. When you provide `prompt_cache_key`, it is combined with the prefix hash, allowing you to influence routing. This is especially beneficial when many requests share long, common prefixes.
363
364Keep the total traffic across all prefixes for each key to approximately 15 requests per minute. If a key receives a higher rate, some requests may miss the cache. For higher-volume workloads, partition traffic across more keys and use a stable mapping so requests with the same key continue to share prefixes.
365
366A cache key improves routing but does not make different prompt prefixes match. Keep the prefix and the cache key consistent across requests that should share cached content.
367
368<a id="prompt-cache-retention"></a>
369
370### Configure prompt cache retention
371
372Use `prompt_cache_retention` to select the retention policy for a supported Responses API or Chat Completions request. Available values depend on the model.
373
374For models that support both in-memory and extended retention, prompt cache pricing is the same for both policies.
375
376**In-memory prompt cache retention**
176 377
177In-memory prompt cache retention is available for models that accept `prompt_cache_retention: "in_memory"`.378In-memory prompt cache retention is available for models that accept `prompt_cache_retention: "in_memory"`.
178 379
179When using the in-memory policy, cached prefixes generally remain active for 5 to 10 minutes of inactivity, up to a maximum of one hour. In-memory cached prefixes are only held within volatile GPU memory.380When using the in-memory policy, cached prefixes generally remain active for 5 to 10 minutes of inactivity, up to a maximum of one hour. In-memory cached prefixes are held only in volatile memory.
381
382<a id="extended-prompt-cache-retention"></a>
180 383
181### Extended prompt cache retention384**Extended prompt cache retention**
385
386Extended prompt cache retention keeps cached prefixes active for longer, up to a maximum of 24 hours.
387
388The 24-hour period is a maximum, not a guarantee that every request will receive a cache hit. Reuse still depends on an exact matching prefix, cache availability, and request routing.
389
390**Models that support extended retention**
182 391
183Extended prompt cache retention is available for the following models:392Extended prompt cache retention is available for the following models:
184 393
195- `gpt-5-codex`404- `gpt-5-codex`
196- `gpt-4.1`405- `gpt-4.1`
197 406
198Extended prompt cache retention keeps cached prefixes active for longer, up to a maximum of 24 hours. Extended Prompt Caching works by offloading the key/value tensors to GPU-local storage when memory is full, significantly increasing the storage capacity available for caching. Note that only key/value tensors are cached in GPU-local storage, not the prompts themselves.407**Retention defaults and Zero Data Retention**
199
200Key/value tensors are the intermediate representation from the model's attention layers produced during prefill. Only the key/value tensors may be persisted in local storage; the original customer content, such as prompt text, is only retained in memory.
201
202### Configure retention for older models
203 408
204For `gpt-5.5` and `gpt-5.5-pro`, only `24h` is supported through `prompt_cache_retention`.409For `gpt-5.5` and `gpt-5.5-pro`, only `24h` is supported through `prompt_cache_retention`.
205 410
206For older models that support both `in_memory` and `24h`, the default depends on your organization's data retention policy:411For models that support both `in_memory` and `24h`, the default depends on your organization's data retention policy:
207
208- Organizations without ZDR enabled default to `24h`.
209- Organizations with ZDR enabled default to `in_memory` when `prompt_cache_retention` is not specified.
210
211The following legacy example sets the retention policy for a `gpt-5.5` request:
212
213```json
214{
215 "model": "gpt-5.5",
216 "input": "Your prompt goes here...",
217 "prompt_cache_retention": "24h"
218}
219```
220
221 412
222## Requirements413- Organizations without Zero Data Retention enabled default to `24h`.
414- Organizations with Zero Data Retention enabled default to `in_memory` when `prompt_cache_retention` is not specified.
223 415
224The minimum prefix length required for caching depends on the model:416Verify the available retention policies for your model and organization before selecting a value.
225 417
226- **GPT-5.6 and later models:** Caching is available for prefixes containing at least 1,024 tokens. This is a strict minimum.418### Measure cache hits and costs
227- **GPT-5.5 and earlier models:** The minimum cacheable prefix length varies by model and can range from 1,024 to 2,048 tokens. Prompts just above 1,024 tokens may not be cached consistently.
228 419
229All requests, including those with fewer than 1,024 tokens, display a `cached_tokens` field in the usage token details. Responses API returns this field in `usage.input_tokens_details` on the [Response object](https://developers.openai.com/api/reference/resources/responses); Chat Completions API returns it in `usage.prompt_tokens_details` on the [Chat object](https://developers.openai.com/api/reference/resources/chat). The field indicates how many input tokens were read from cache. For requests under 1,024 tokens, `cached_tokens` is zero.420Use `cached_tokens` to see how many input tokens were read from the cache. The field is present even when no tokens were cached.
230 421
231For GPT-5.6 models and later model families, `cache_write_tokens` reports the number of prompt tokens written to cache. Cache write billing uses this value at 1.25× the uncached input token rate.422For the Responses API, the field appears in `usage.input_tokens_details.cached_tokens`. For the Chat Completions API, it appears in `usage.prompt_tokens_details.cached_tokens`.
232 423
233The following Chat Completions usage example shows both fields. In this response, 1,920 tokens were read from cache and no tokens were written:424The following Chat Completions usage example shows a request that reused 1,920 of its 2,006 prompt tokens:
234 425
235```json426```json
236"usage": {427{
428 "usage": {
237 "prompt_tokens": 2006,429 "prompt_tokens": 2006,
238 "completion_tokens": 300,430 "completion_tokens": 300,
239 "total_tokens": 2306,431 "total_tokens": 2306,
240 "prompt_tokens_details": {432 "prompt_tokens_details": {
241 "cached_tokens": 1920,433 "cached_tokens": 1920
242 "cache_write_tokens": 0434 }
243 },
244 "completion_tokens_details": {
245 "reasoning_tokens": 0,
246 "accepted_prediction_tokens": 0,
247 "rejected_prediction_tokens": 0
248 }435 }
249}436}
250```437```
251 438
252### What can be cached439In this example, the remaining 86 prompt tokens were not read from the cache. Monitor cached-token usage across requests to identify changes in prompt structure, traffic patterns, or cache availability.
253 440
254- **Messages:** The complete messages array, encompassing system, user, and assistant interactions.441**Pricing and rate limits**
255- **Images:** Images included in user messages, either as links or as base64-encoded data, as well as multiple images can be sent. Ensure the detail parameter is set identically, as it impacts image tokenization.442
256- **Tool use:** Both the messages array and the list of available `tools` can be cached, contributing to the model's minimum cacheable prefix length.443Creating a cache entry has no additional fee. Cached input is billed at the cached-input rate when the model offers one. Rates and discounts vary by model.
257- **Structured outputs:** The structured output schema serves as a prefix to the system message and can be cached.444
445Cached input tokens still count toward tokens-per-minute rate limits. Prompt caching does not change rate-limit calculations or guarantee identical model outputs.
446
447### What can be cached
258 448
259## Best practices449- **Messages:** System, developer, user, and assistant messages can contribute to a reusable prompt prefix.
450- **Images:** Image inputs can be cached when the images, their order, and their detail settings remain the same.
451- **Tools:** Tool definitions, descriptions, parameter schemas, and tool ordering can contribute to the prefix.
452- **Structured outputs:** A structured output schema can be included in the reusable prompt prefix.
453- **Audio:** Supported audio inputs can contribute to cacheable prompt content.
260 454
261- Structure prompts with **static or repeated content at the beginning** and dynamic, user-specific content at the end.455All reusable content must remain identical across requests. Changes earlier in the prompt can invalidate reuse for the content that follows.
262- Use the **[`prompt_cache_key`](https://developers.openai.com/api/reference/resources/responses/methods/create#responses-create-prompt_cache_key) parameter** consistently across requests that share long, common prefixes to improve cache hit rates. On GPT-5.6 models and later model families, you must set this parameter to use the more reliable cache matching. Keep the total traffic for each key to approximately 15 requests per minute, and use more keys for higher-volume workloads.
263- On GPT-5.6 models and later model families, place **explicit cache breakpoints** after stable prompt content that is likely to be reused. Set `prompt_cache_options.mode` to `explicit` when you want the service to use only the breakpoints you provide.
264- **Monitor cache reads and writes** by logging `cached_tokens` and `cache_write_tokens`. Compare cache-write volume with subsequent cache reads to understand net cost and adjust breakpoint placement. You can also monitor cached token counts in the OpenAI Usage dashboard.
265- **Maintain a steady stream of requests** with identical prompt prefixes to minimize cache evictions and maximize caching benefits.
266 456
267## Frequently asked questions457## Frequently asked questions
268 458
276 466
2773. **Is there a way to manually clear the cache?**4673. **Is there a way to manually clear the cache?**
278 468
279 Manual cache clearing is not currently available. For models before the GPT-5.6 family that use in-memory retention, typical cache evictions occur after 5-10 minutes of inactivity, though entries can remain for up to one hour during off-peak periods. For GPT-5.6 models and later model families, cached prefixes remain eligible for reuse for at least 30 minutes and may be retained longer.469 Manual cache clearing is not currently available. For models before the GPT-5.6 family that use in-memory retention, typical cache evictions occur after 5-10 minutes of inactivity, though entries can remain for up to one hour during off-peak periods. For GPT-5.6 models and later model families, cached prefixes remain eligible for reuse for 30 minutes and may be retained longer.
280 470
2814. **Will I be expected to pay extra for writing to Prompt Caching?**4714. **Will I be expected to pay extra for writing to Prompt Caching?**
282 472
