diff --git a/en/resources/evals/subresources/runs/index.md b/en/resources/evals/subresources/runs/index.md index 8810586..dd3749a 100644 --- a/en/resources/evals/subresources/runs/index.md +++ b/en/resources/evals/subresources/runs/index.md @@ -4,7 +4,7 @@ **get** `/evals/{eval_id}/runs` -Get a list of runs for an evaluation. +Get eval runs ### Path Parameters @@ -68,7 +68,7 @@ Get a list of runs for an evaluation. Determines what populates the `item` namespace in the data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -84,7 +84,7 @@ Get a list of runs for an evaluation. - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -110,7 +110,7 @@ Get a list of runs for an evaluation. Determines what populates the `item` namespace in this run's data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -126,7 +126,7 @@ Get a list of runs for an evaluation. - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -138,7 +138,7 @@ Get a list of runs for an evaluation. - `"file_id"` - - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }` + - `StoredCompletions object { type, created_after, created_before, 3 more }` A StoredCompletionsRunDataSource configuration describing a set of filters @@ -183,7 +183,7 @@ Get a list of runs for an evaluation. Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - - `TemplateInputMessages object { template, type }` + - `Template object { template, type }` - `template: array of EasyInputMessage or object { content, role, type }` @@ -211,7 +211,7 @@ Get a list of runs for an evaluation. A list of one or many input items to the model, containing different content types. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -225,9 +225,19 @@ Get a list of runs for an evaluation. - `"input_text"` - - `ResponseInputImage object { detail, type, file_id, image_url }` + - `prompt_cache_breakpoint: optional object { mode }` - An image input to the model. Learn about [image inputs](/docs/guides/vision). + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + + - `ResponseInputImage object { detail, type, file_id, 2 more }` + + An image input to the model. Learn about [image inputs](https://platform.openai.com/docs/guides/vision). - `detail: "low" or "high" or "auto" or "original"` @@ -255,7 +265,17 @@ Get a list of runs for an evaluation. The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL. - - `ResponseInputFile object { type, detail, file_data, 3 more }` + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + + - `ResponseInputFile object { type, detail, file_data, 4 more }` A file input to the model. @@ -265,9 +285,11 @@ Get a list of runs for an evaluation. - `"input_file"` - - `detail: optional "low" or "high"` + - `detail: optional "auto" or "low" or "high"` - The detail level of the file to be sent to the model. Use `low` for the default rendering behavior, or `high` to render the file at higher quality. Defaults to `low`. + The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`. + + - `"auto"` - `"low"` @@ -289,6 +311,16 @@ Get a list of runs for an evaluation. The name of the file to be sent to the model. + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or @@ -318,7 +350,7 @@ Get a list of runs for an evaluation. - `"message"` - - `EvalMessageObject object { content, role, type }` + - `EvalItem object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take @@ -334,7 +366,7 @@ Get a list of runs for an evaluation. A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -404,7 +436,7 @@ Get a list of runs for an evaluation. A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -469,7 +501,7 @@ Get a list of runs for an evaluation. - `"template"` - - `ItemReferenceInputMessages object { item_reference, type }` + - `ItemReference object { item_reference, type }` - `item_reference: string` @@ -493,16 +525,13 @@ Get a list of runs for an evaluation. - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `"none"` @@ -516,6 +545,8 @@ Get a list of runs for an evaluation. - `"xhigh"` + - `"max"` + - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject` An object specifying the format that the model must output. @@ -523,7 +554,7 @@ Get a list of runs for an evaluation. Setting to `{ "type": "json_schema", "json_schema": {...} }` enables Structured Outputs which ensures the model will match your supplied JSON schema. Learn more in the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` @@ -542,7 +573,7 @@ Get a list of runs for an evaluation. - `ResponseFormatJSONSchema object { json_schema, type }` JSON Schema response format. Used to generate structured JSON responses. - Learn more about [Structured Outputs](/docs/guides/structured-outputs). + Learn more about [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs). - `json_schema: object { name, description, schema, strict }` @@ -569,7 +600,7 @@ Get a list of runs for an evaluation. If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). - `type: "json_schema"` @@ -614,13 +645,13 @@ Get a list of runs for an evaluation. - `parameters: optional FunctionParameters` - The parameters the functions accepts, described as a JSON Schema object. See the [guide](/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. + The parameters the functions accepts, described as a JSON Schema object. See the [guide](https://platform.openai.com/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. Omitting `parameters` defines a function with an empty parameter list. - `strict: optional boolean` - Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/docs/guides/function-calling). + Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](https://platform.openai.com/docs/guides/function-calling). - `type: "function"` @@ -632,7 +663,7 @@ Get a list of runs for an evaluation. An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `ResponsesRunDataSource object { source, type, input_messages, 2 more }` + - `Responses object { source, type, input_messages, 2 more }` A ResponsesRunDataSource object describing a model sampling configuration. @@ -640,7 +671,7 @@ Get a list of runs for an evaluation. Determines what populates the `item` namespace in this run's data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -656,7 +687,7 @@ Get a list of runs for an evaluation. - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -668,7 +699,7 @@ Get a list of runs for an evaluation. - `"file_id"` - - `EvalResponsesSource object { type, created_after, created_before, 8 more }` + - `Responses object { type, created_after, created_before, 8 more }` A EvalResponsesSource object describing a run data source configuration. @@ -700,16 +731,13 @@ Get a list of runs for an evaluation. - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `temperature: optional number` @@ -737,7 +765,7 @@ Get a list of runs for an evaluation. Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - - `InputMessagesTemplate object { template, type }` + - `Template object { template, type }` - `template: array of object { content, role } or object { content, role, type }` @@ -753,7 +781,7 @@ Get a list of runs for an evaluation. The role of the message (e.g. "system", "assistant", "user"). - - `EvalMessageObject object { content, role, type }` + - `EvalItem object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take @@ -769,7 +797,7 @@ Get a list of runs for an evaluation. A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -839,7 +867,7 @@ Get a list of runs for an evaluation. - `"template"` - - `InputMessagesItemReference object { item_reference, type }` + - `ItemReference object { item_reference, type }` - `item_reference: string` @@ -863,16 +891,13 @@ Get a list of runs for an evaluation. - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `seed: optional number` @@ -887,8 +912,8 @@ Get a list of runs for an evaluation. Configuration options for a text response from the model. Can be plain text or structured JSON data. Learn more: - - [Text inputs and outputs](/docs/guides/text) - - [Structured Outputs](/docs/guides/structured-outputs) + - [Text inputs and outputs](https://platform.openai.com/docs/guides/text) + - [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs) - `format: optional ResponseFormatTextConfig` @@ -896,7 +921,7 @@ Get a list of runs for an evaluation. Configuring `{ "type": "json_schema" }` enables Structured Outputs, which ensures the model will match your supplied JSON schema. Learn more in the - [Structured Outputs guide](/docs/guides/structured-outputs). + [Structured Outputs guide](https://platform.openai.com/docs/guides/structured-outputs). The default format is `{ "type": "text" }` with no additional options. @@ -913,7 +938,7 @@ Get a list of runs for an evaluation. - `ResponseFormatTextJSONSchemaConfig object { name, schema, type, 2 more }` JSON Schema response format. Used to generate structured JSON responses. - Learn more about [Structured Outputs](/docs/guides/structured-outputs). + Learn more about [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs). - `name: string` @@ -942,7 +967,7 @@ Get a list of runs for an evaluation. If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). - `ResponseFormatJSONObject object { type }` @@ -951,7 +976,7 @@ Get a list of runs for an evaluation. model will not generate JSON without a system or user message instructing it to do so. - - `tools: optional array of object { name, parameters, strict, 3 more } or object { type, vector_store_ids, filters, 2 more } or object { type } or 12 more` + - `tools: optional array of object { name, parameters, strict, 5 more } or object { type, vector_store_ids, filters, 2 more } or object { type } or 13 more` An array of tools the model may call while generating a response. You can specify which tool to use by setting the `tool_choice` parameter. @@ -959,14 +984,14 @@ Get a list of runs for an evaluation. The two categories of tools you can provide the model are: - **Built-in tools**: Tools that are provided by OpenAI that extend the - model's capabilities, like [web search](/docs/guides/tools-web-search) - or [file search](/docs/guides/tools-file-search). Learn more about - [built-in tools](/docs/guides/tools). + model's capabilities, like [web search](https://platform.openai.com/docs/guides/tools-web-search) + or [file search](https://platform.openai.com/docs/guides/tools-file-search). Learn more about + [built-in tools](https://platform.openai.com/docs/guides/tools). - **Function calls (custom tools)**: Functions that are defined by you, enabling the model to call your own code. Learn more about - [function calling](/docs/guides/function-calling). + [function calling](https://platform.openai.com/docs/guides/function-calling). - - `Function object { name, parameters, strict, 3 more }` + - `Function object { name, parameters, strict, 5 more }` Defines a function in your own code the model can choose to call. Learn more about [function calling](https://platform.openai.com/docs/guides/function-calling). @@ -980,7 +1005,7 @@ Get a list of runs for an evaluation. - `strict: boolean` - Whether to enforce strict parameter validation. Default `true`. + Whether strict parameter validation is enforced for this function tool. - `type: "function"` @@ -988,6 +1013,14 @@ Get a list of runs for an evaluation. - `"function"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this function is deferred and loaded via tool search. @@ -996,6 +1029,10 @@ Get a list of runs for an evaluation. A description of the function. Used by the model to determine whether or not to call the function. + - `output_schema: optional map[unknown]` + + A JSON schema object describing the JSON value encoded in string outputs for this function. + - `FileSearch object { type, vector_store_ids, filters, 2 more }` A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](https://platform.openai.com/docs/guides/tools-file-search). @@ -1166,7 +1203,7 @@ Get a list of runs for an evaluation. - `WebSearch object { type, filters, search_context_size, user_location }` Search the Internet for sources related to the prompt. Learn more about the - [web search tool](/docs/guides/tools-web-search). + [web search tool](https://platform.openai.com/docs/guides/tools-web-search). - `type: "web_search" or "web_search_2025_08_26"` @@ -1223,10 +1260,10 @@ Get a list of runs for an evaluation. - `"approximate"` - - `Mcp object { server_label, type, allowed_tools, 8 more }` + - `Mcp object { server_label, type, allowed_callers, 9 more }` Give the model access to additional tools via remote Model Context Protocol - (MCP) servers. [Learn more about MCP](/docs/guides/tools-remote-mcp). + (MCP) servers. [Learn more about MCP](https://platform.openai.com/docs/guides/tools-remote-mcp). - `server_label: string` @@ -1238,6 +1275,14 @@ Get a list of runs for an evaluation. - `"mcp"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `allowed_tools: optional array of string or object { read_only, tool_names }` List of allowed tool names or a filter object. @@ -1270,7 +1315,7 @@ Get a list of runs for an evaluation. Identifier for service connectors, like those available in ChatGPT. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more - about service connectors [here](/docs/guides/tools-remote-mcp#connectors). + about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors). Currently supported `connector_id` values are: @@ -1370,7 +1415,7 @@ Get a list of runs for an evaluation. The Secure MCP Tunnel ID to use instead of a direct server URL. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. - - `CodeInterpreter object { container, type }` + - `CodeInterpreter object { container, type, allowed_callers }` A tool that runs Python code to help generate a response to a prompt. @@ -1456,6 +1501,22 @@ Get a list of runs for an evaluation. - `"code_interpreter"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + + - `ProgrammaticToolCalling object { type }` + + - `type: "programmatic_tool_calling"` + + The type of the tool. Always `programmatic_tool_calling`. + + - `"programmatic_tool_calling"` + - `ImageGeneration object { type, action, background, 9 more }` A tool that generates images using the GPT image models. @@ -1478,8 +1539,19 @@ Get a list of runs for an evaluation. - `background: optional "transparent" or "opaque" or "auto"` - Background type for the generated image. One of `transparent`, - `opaque`, or `auto`. Default: `auto`. + Allows to set transparency for the background of the generated image(s). + This parameter is only supported for GPT image models that support + transparent backgrounds. Must be one of `transparent`, `opaque`, or + `auto` (default value). When `auto` is used, the model will + automatically determine the best background for the image. + + `gpt-image-2` and `gpt-image-2-2026-04-21` do not support + transparent backgrounds. Requests with `background` set to + `transparent` will return an error for these models; use `opaque` or + `auto` instead. + + If `transparent`, the output format needs to support transparency, + so it should be set to either `png` (default value) or `webp`. - `"transparent"` @@ -1508,13 +1580,13 @@ Get a list of runs for an evaluation. Base64-encoded mask image. - - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5"` + - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-2" or 3 more` The image generation model to use. Default: `gpt-image-1`. - `string` - - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5"` + - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-2" or 3 more` The image generation model to use. Default: `gpt-image-1`. @@ -1522,8 +1594,14 @@ Get a list of runs for an evaluation. - `"gpt-image-1-mini"` + - `"gpt-image-2"` + + - `"gpt-image-2-2026-04-21"` + - `"gpt-image-1.5"` + - `"chatgpt-image-latest"` + - `moderation: optional "auto" or "low"` Moderation level for the generated image. Default: `auto`. @@ -1592,7 +1670,7 @@ Get a list of runs for an evaluation. - `"local_shell"` - - `Shell object { type, environment }` + - `Shell object { type, allowed_callers, environment }` A tool that allows the model to execute shell commands. @@ -1602,6 +1680,14 @@ Get a list of runs for an evaluation. - `"shell"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `environment: optional ContainerAuto or LocalEnvironment or ContainerReference` - `ContainerAuto object { type, file_ids, memory_limit, 2 more }` @@ -1728,9 +1814,9 @@ Get a list of runs for an evaluation. - `"container_reference"` - - `Custom object { name, type, defer_loading, 2 more }` + - `Custom object { name, type, allowed_callers, 3 more }` - A custom tool that processes input using a specified format. Learn more about [custom tools](/docs/guides/function-calling#custom-tools) + A custom tool that processes input using a specified format. Learn more about [custom tools](https://platform.openai.com/docs/guides/function-calling#custom-tools) - `name: string` @@ -1742,6 +1828,14 @@ Get a list of runs for an evaluation. - `"custom"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this tool should be deferred and discovered via tool search. @@ -1798,11 +1892,11 @@ Get a list of runs for an evaluation. The namespace name used in tool calls (for example, `crm`). - - `tools: array of object { name, type, defer_loading, 3 more } or object { name, type, defer_loading, 2 more }` + - `tools: array of object { name, type, allowed_callers, 5 more } or object { name, type, allowed_callers, 3 more }` The function/custom tools available inside this namespace. - - `Function object { name, type, defer_loading, 3 more }` + - `Function object { name, type, allowed_callers, 5 more }` - `name: string` @@ -1810,19 +1904,33 @@ Get a list of runs for an evaluation. - `"function"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this function should be deferred and discovered via tool search. - `description: optional string` + - `output_schema: optional map[unknown]` + + A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs. + - `parameters: optional unknown` - `strict: optional boolean` - - `Custom object { name, type, defer_loading, 2 more }` + Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise. - A custom tool that processes input using a specified format. Learn more about [custom tools](/docs/guides/function-calling#custom-tools) + - `Custom object { name, type, allowed_callers, 3 more }` + + A custom tool that processes input using a specified format. Learn more about [custom tools](https://platform.openai.com/docs/guides/function-calling#custom-tools) - `name: string` @@ -1834,6 +1942,14 @@ Get a list of runs for an evaluation. - `"custom"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this tool should be deferred and discovered via tool search. @@ -1932,7 +2048,7 @@ Get a list of runs for an evaluation. The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`. - - `ApplyPatch object { type }` + - `ApplyPatch object { type, allowed_callers }` Allows the assistant to create, delete, or update files using unified diffs. @@ -1942,6 +2058,14 @@ Get a list of runs for an evaluation. - `"apply_patch"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `top_p: optional number` An alternative to temperature for nucleus sampling; 1.0 includes all tokens. @@ -2247,7 +2371,7 @@ curl https://api.openai.com/v1/evals/egroup_67abd54d9b0081909a86353f6fb9317a/run **post** `/evals/{eval_id}/runs` -Kicks off a new run for a given evaluation, specifying the data source, and what model configuration to use to test. The datasource will be validated against the schema specified in the config of the evaluation. +Create eval run ### Path Parameters @@ -2267,7 +2391,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what Determines what populates the `item` namespace in the data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -2283,7 +2407,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -2309,7 +2433,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what Determines what populates the `item` namespace in this run's data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -2325,7 +2449,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -2337,7 +2461,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"file_id"` - - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }` + - `StoredCompletions object { type, created_after, created_before, 3 more }` A StoredCompletionsRunDataSource configuration describing a set of filters @@ -2382,7 +2506,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - - `TemplateInputMessages object { template, type }` + - `Template object { template, type }` - `template: array of EasyInputMessage or object { content, role, type }` @@ -2410,7 +2534,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what A list of one or many input items to the model, containing different content types. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -2424,9 +2548,19 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"input_text"` - - `ResponseInputImage object { detail, type, file_id, image_url }` + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + + - `ResponseInputImage object { detail, type, file_id, 2 more }` - An image input to the model. Learn about [image inputs](/docs/guides/vision). + An image input to the model. Learn about [image inputs](https://platform.openai.com/docs/guides/vision). - `detail: "low" or "high" or "auto" or "original"` @@ -2454,7 +2588,17 @@ Kicks off a new run for a given evaluation, specifying the data source, and what The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL. - - `ResponseInputFile object { type, detail, file_data, 3 more }` + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + + - `ResponseInputFile object { type, detail, file_data, 4 more }` A file input to the model. @@ -2464,9 +2608,11 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"input_file"` - - `detail: optional "low" or "high"` + - `detail: optional "auto" or "low" or "high"` + + The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`. - The detail level of the file to be sent to the model. Use `low` for the default rendering behavior, or `high` to render the file at higher quality. Defaults to `low`. + - `"auto"` - `"low"` @@ -2488,6 +2634,16 @@ Kicks off a new run for a given evaluation, specifying the data source, and what The name of the file to be sent to the model. + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or @@ -2517,7 +2673,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"message"` - - `EvalMessageObject object { content, role, type }` + - `EvalItem object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take @@ -2533,7 +2689,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -2603,7 +2759,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -2668,7 +2824,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"template"` - - `ItemReferenceInputMessages object { item_reference, type }` + - `ItemReference object { item_reference, type }` - `item_reference: string` @@ -2692,16 +2848,13 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `"none"` @@ -2715,6 +2868,8 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"xhigh"` + - `"max"` + - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject` An object specifying the format that the model must output. @@ -2722,7 +2877,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what Setting to `{ "type": "json_schema", "json_schema": {...} }` enables Structured Outputs which ensures the model will match your supplied JSON schema. Learn more in the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` @@ -2741,7 +2896,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `ResponseFormatJSONSchema object { json_schema, type }` JSON Schema response format. Used to generate structured JSON responses. - Learn more about [Structured Outputs](/docs/guides/structured-outputs). + Learn more about [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs). - `json_schema: object { name, description, schema, strict }` @@ -2768,7 +2923,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). - `type: "json_schema"` @@ -2813,13 +2968,13 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `parameters: optional FunctionParameters` - The parameters the functions accepts, described as a JSON Schema object. See the [guide](/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. + The parameters the functions accepts, described as a JSON Schema object. See the [guide](https://platform.openai.com/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. Omitting `parameters` defines a function with an empty parameter list. - `strict: optional boolean` - Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/docs/guides/function-calling). + Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](https://platform.openai.com/docs/guides/function-calling). - `type: "function"` @@ -2831,7 +2986,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `ResponsesRunDataSource object { source, type, input_messages, 2 more }` + - `CreateEvalResponsesRunDataSource object { source, type, input_messages, 2 more }` A ResponsesRunDataSource object describing a model sampling configuration. @@ -2839,7 +2994,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what Determines what populates the `item` namespace in this run's data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -2855,7 +3010,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -2867,7 +3022,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"file_id"` - - `EvalResponsesSource object { type, created_after, created_before, 8 more }` + - `Responses object { type, created_after, created_before, 8 more }` A EvalResponsesSource object describing a run data source configuration. @@ -2899,16 +3054,13 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `temperature: optional number` @@ -2936,7 +3088,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - - `InputMessagesTemplate object { template, type }` + - `Template object { template, type }` - `template: array of object { content, role } or object { content, role, type }` @@ -2952,7 +3104,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what The role of the message (e.g. "system", "assistant", "user"). - - `EvalMessageObject object { content, role, type }` + - `EvalItem object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take @@ -2968,7 +3120,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -3038,7 +3190,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"template"` - - `InputMessagesItemReference object { item_reference, type }` + - `ItemReference object { item_reference, type }` - `item_reference: string` @@ -3062,16 +3214,13 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `seed: optional number` @@ -3086,8 +3235,8 @@ Kicks off a new run for a given evaluation, specifying the data source, and what Configuration options for a text response from the model. Can be plain text or structured JSON data. Learn more: - - [Text inputs and outputs](/docs/guides/text) - - [Structured Outputs](/docs/guides/structured-outputs) + - [Text inputs and outputs](https://platform.openai.com/docs/guides/text) + - [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs) - `format: optional ResponseFormatTextConfig` @@ -3095,7 +3244,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what Configuring `{ "type": "json_schema" }` enables Structured Outputs, which ensures the model will match your supplied JSON schema. Learn more in the - [Structured Outputs guide](/docs/guides/structured-outputs). + [Structured Outputs guide](https://platform.openai.com/docs/guides/structured-outputs). The default format is `{ "type": "text" }` with no additional options. @@ -3112,7 +3261,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `ResponseFormatTextJSONSchemaConfig object { name, schema, type, 2 more }` JSON Schema response format. Used to generate structured JSON responses. - Learn more about [Structured Outputs](/docs/guides/structured-outputs). + Learn more about [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs). - `name: string` @@ -3141,7 +3290,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). - `ResponseFormatJSONObject object { type }` @@ -3150,7 +3299,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what model will not generate JSON without a system or user message instructing it to do so. - - `tools: optional array of object { name, parameters, strict, 3 more } or object { type, vector_store_ids, filters, 2 more } or object { type } or 12 more` + - `tools: optional array of object { name, parameters, strict, 5 more } or object { type, vector_store_ids, filters, 2 more } or object { type } or 13 more` An array of tools the model may call while generating a response. You can specify which tool to use by setting the `tool_choice` parameter. @@ -3158,14 +3307,14 @@ Kicks off a new run for a given evaluation, specifying the data source, and what The two categories of tools you can provide the model are: - **Built-in tools**: Tools that are provided by OpenAI that extend the - model's capabilities, like [web search](/docs/guides/tools-web-search) - or [file search](/docs/guides/tools-file-search). Learn more about - [built-in tools](/docs/guides/tools). + model's capabilities, like [web search](https://platform.openai.com/docs/guides/tools-web-search) + or [file search](https://platform.openai.com/docs/guides/tools-file-search). Learn more about + [built-in tools](https://platform.openai.com/docs/guides/tools). - **Function calls (custom tools)**: Functions that are defined by you, enabling the model to call your own code. Learn more about - [function calling](/docs/guides/function-calling). + [function calling](https://platform.openai.com/docs/guides/function-calling). - - `Function object { name, parameters, strict, 3 more }` + - `Function object { name, parameters, strict, 5 more }` Defines a function in your own code the model can choose to call. Learn more about [function calling](https://platform.openai.com/docs/guides/function-calling). @@ -3179,7 +3328,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `strict: boolean` - Whether to enforce strict parameter validation. Default `true`. + Whether strict parameter validation is enforced for this function tool. - `type: "function"` @@ -3187,6 +3336,14 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"function"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this function is deferred and loaded via tool search. @@ -3195,6 +3352,10 @@ Kicks off a new run for a given evaluation, specifying the data source, and what A description of the function. Used by the model to determine whether or not to call the function. + - `output_schema: optional map[unknown]` + + A JSON schema object describing the JSON value encoded in string outputs for this function. + - `FileSearch object { type, vector_store_ids, filters, 2 more }` A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](https://platform.openai.com/docs/guides/tools-file-search). @@ -3365,7 +3526,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `WebSearch object { type, filters, search_context_size, user_location }` Search the Internet for sources related to the prompt. Learn more about the - [web search tool](/docs/guides/tools-web-search). + [web search tool](https://platform.openai.com/docs/guides/tools-web-search). - `type: "web_search" or "web_search_2025_08_26"` @@ -3422,10 +3583,10 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"approximate"` - - `Mcp object { server_label, type, allowed_tools, 8 more }` + - `Mcp object { server_label, type, allowed_callers, 9 more }` Give the model access to additional tools via remote Model Context Protocol - (MCP) servers. [Learn more about MCP](/docs/guides/tools-remote-mcp). + (MCP) servers. [Learn more about MCP](https://platform.openai.com/docs/guides/tools-remote-mcp). - `server_label: string` @@ -3437,6 +3598,14 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"mcp"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `allowed_tools: optional array of string or object { read_only, tool_names }` List of allowed tool names or a filter object. @@ -3469,7 +3638,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what Identifier for service connectors, like those available in ChatGPT. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more - about service connectors [here](/docs/guides/tools-remote-mcp#connectors). + about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors). Currently supported `connector_id` values are: @@ -3569,7 +3738,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what The Secure MCP Tunnel ID to use instead of a direct server URL. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. - - `CodeInterpreter object { container, type }` + - `CodeInterpreter object { container, type, allowed_callers }` A tool that runs Python code to help generate a response to a prompt. @@ -3655,6 +3824,22 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"code_interpreter"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + + - `ProgrammaticToolCalling object { type }` + + - `type: "programmatic_tool_calling"` + + The type of the tool. Always `programmatic_tool_calling`. + + - `"programmatic_tool_calling"` + - `ImageGeneration object { type, action, background, 9 more }` A tool that generates images using the GPT image models. @@ -3677,8 +3862,19 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `background: optional "transparent" or "opaque" or "auto"` - Background type for the generated image. One of `transparent`, - `opaque`, or `auto`. Default: `auto`. + Allows to set transparency for the background of the generated image(s). + This parameter is only supported for GPT image models that support + transparent backgrounds. Must be one of `transparent`, `opaque`, or + `auto` (default value). When `auto` is used, the model will + automatically determine the best background for the image. + + `gpt-image-2` and `gpt-image-2-2026-04-21` do not support + transparent backgrounds. Requests with `background` set to + `transparent` will return an error for these models; use `opaque` or + `auto` instead. + + If `transparent`, the output format needs to support transparency, + so it should be set to either `png` (default value) or `webp`. - `"transparent"` @@ -3707,13 +3903,13 @@ Kicks off a new run for a given evaluation, specifying the data source, and what Base64-encoded mask image. - - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5"` + - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-2" or 3 more` The image generation model to use. Default: `gpt-image-1`. - `string` - - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5"` + - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-2" or 3 more` The image generation model to use. Default: `gpt-image-1`. @@ -3721,8 +3917,14 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"gpt-image-1-mini"` + - `"gpt-image-2"` + + - `"gpt-image-2-2026-04-21"` + - `"gpt-image-1.5"` + - `"chatgpt-image-latest"` + - `moderation: optional "auto" or "low"` Moderation level for the generated image. Default: `auto`. @@ -3791,7 +3993,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"local_shell"` - - `Shell object { type, environment }` + - `Shell object { type, allowed_callers, environment }` A tool that allows the model to execute shell commands. @@ -3801,6 +4003,14 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"shell"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `environment: optional ContainerAuto or LocalEnvironment or ContainerReference` - `ContainerAuto object { type, file_ids, memory_limit, 2 more }` @@ -3927,9 +4137,9 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"container_reference"` - - `Custom object { name, type, defer_loading, 2 more }` + - `Custom object { name, type, allowed_callers, 3 more }` - A custom tool that processes input using a specified format. Learn more about [custom tools](/docs/guides/function-calling#custom-tools) + A custom tool that processes input using a specified format. Learn more about [custom tools](https://platform.openai.com/docs/guides/function-calling#custom-tools) - `name: string` @@ -3941,6 +4151,14 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"custom"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this tool should be deferred and discovered via tool search. @@ -3997,11 +4215,11 @@ Kicks off a new run for a given evaluation, specifying the data source, and what The namespace name used in tool calls (for example, `crm`). - - `tools: array of object { name, type, defer_loading, 3 more } or object { name, type, defer_loading, 2 more }` + - `tools: array of object { name, type, allowed_callers, 5 more } or object { name, type, allowed_callers, 3 more }` The function/custom tools available inside this namespace. - - `Function object { name, type, defer_loading, 3 more }` + - `Function object { name, type, allowed_callers, 5 more }` - `name: string` @@ -4009,19 +4227,33 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"function"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this function should be deferred and discovered via tool search. - `description: optional string` + - `output_schema: optional map[unknown]` + + A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs. + - `parameters: optional unknown` - `strict: optional boolean` - - `Custom object { name, type, defer_loading, 2 more }` + Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise. + + - `Custom object { name, type, allowed_callers, 3 more }` - A custom tool that processes input using a specified format. Learn more about [custom tools](/docs/guides/function-calling#custom-tools) + A custom tool that processes input using a specified format. Learn more about [custom tools](https://platform.openai.com/docs/guides/function-calling#custom-tools) - `name: string` @@ -4033,6 +4265,14 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"custom"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this tool should be deferred and discovered via tool search. @@ -4131,7 +4371,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`. - - `ApplyPatch object { type }` + - `ApplyPatch object { type, allowed_callers }` Allows the assistant to create, delete, or update files using unified diffs. @@ -4141,6 +4381,14 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"apply_patch"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `top_p: optional number` An alternative to temperature for nucleus sampling; 1.0 includes all tokens. @@ -4180,7 +4428,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what Determines what populates the `item` namespace in the data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -4196,7 +4444,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -4222,7 +4470,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what Determines what populates the `item` namespace in this run's data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -4238,7 +4486,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -4250,7 +4498,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"file_id"` - - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }` + - `StoredCompletions object { type, created_after, created_before, 3 more }` A StoredCompletionsRunDataSource configuration describing a set of filters @@ -4295,7 +4543,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - - `TemplateInputMessages object { template, type }` + - `Template object { template, type }` - `template: array of EasyInputMessage or object { content, role, type }` @@ -4323,7 +4571,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what A list of one or many input items to the model, containing different content types. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -4337,9 +4585,19 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"input_text"` - - `ResponseInputImage object { detail, type, file_id, image_url }` + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` - An image input to the model. Learn about [image inputs](/docs/guides/vision). + The breakpoint mode. Always `explicit`. + + - `"explicit"` + + - `ResponseInputImage object { detail, type, file_id, 2 more }` + + An image input to the model. Learn about [image inputs](https://platform.openai.com/docs/guides/vision). - `detail: "low" or "high" or "auto" or "original"` @@ -4367,7 +4625,17 @@ Kicks off a new run for a given evaluation, specifying the data source, and what The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL. - - `ResponseInputFile object { type, detail, file_data, 3 more }` + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + + - `ResponseInputFile object { type, detail, file_data, 4 more }` A file input to the model. @@ -4377,9 +4645,11 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"input_file"` - - `detail: optional "low" or "high"` + - `detail: optional "auto" or "low" or "high"` - The detail level of the file to be sent to the model. Use `low` for the default rendering behavior, or `high` to render the file at higher quality. Defaults to `low`. + The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`. + + - `"auto"` - `"low"` @@ -4401,6 +4671,16 @@ Kicks off a new run for a given evaluation, specifying the data source, and what The name of the file to be sent to the model. + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or @@ -4430,7 +4710,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"message"` - - `EvalMessageObject object { content, role, type }` + - `EvalItem object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take @@ -4446,7 +4726,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -4516,7 +4796,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -4581,7 +4861,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"template"` - - `ItemReferenceInputMessages object { item_reference, type }` + - `ItemReference object { item_reference, type }` - `item_reference: string` @@ -4605,16 +4885,13 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `"none"` @@ -4628,6 +4905,8 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"xhigh"` + - `"max"` + - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject` An object specifying the format that the model must output. @@ -4635,7 +4914,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what Setting to `{ "type": "json_schema", "json_schema": {...} }` enables Structured Outputs which ensures the model will match your supplied JSON schema. Learn more in the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` @@ -4654,7 +4933,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `ResponseFormatJSONSchema object { json_schema, type }` JSON Schema response format. Used to generate structured JSON responses. - Learn more about [Structured Outputs](/docs/guides/structured-outputs). + Learn more about [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs). - `json_schema: object { name, description, schema, strict }` @@ -4681,7 +4960,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). - `type: "json_schema"` @@ -4726,13 +5005,13 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `parameters: optional FunctionParameters` - The parameters the functions accepts, described as a JSON Schema object. See the [guide](/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. + The parameters the functions accepts, described as a JSON Schema object. See the [guide](https://platform.openai.com/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. Omitting `parameters` defines a function with an empty parameter list. - `strict: optional boolean` - Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/docs/guides/function-calling). + Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](https://platform.openai.com/docs/guides/function-calling). - `type: "function"` @@ -4744,7 +5023,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `ResponsesRunDataSource object { source, type, input_messages, 2 more }` + - `Responses object { source, type, input_messages, 2 more }` A ResponsesRunDataSource object describing a model sampling configuration. @@ -4752,7 +5031,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what Determines what populates the `item` namespace in this run's data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -4768,7 +5047,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -4780,7 +5059,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"file_id"` - - `EvalResponsesSource object { type, created_after, created_before, 8 more }` + - `Responses object { type, created_after, created_before, 8 more }` A EvalResponsesSource object describing a run data source configuration. @@ -4812,16 +5091,13 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `temperature: optional number` @@ -4849,7 +5125,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - - `InputMessagesTemplate object { template, type }` + - `Template object { template, type }` - `template: array of object { content, role } or object { content, role, type }` @@ -4865,7 +5141,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what The role of the message (e.g. "system", "assistant", "user"). - - `EvalMessageObject object { content, role, type }` + - `EvalItem object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take @@ -4881,7 +5157,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -4951,7 +5227,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"template"` - - `InputMessagesItemReference object { item_reference, type }` + - `ItemReference object { item_reference, type }` - `item_reference: string` @@ -4975,16 +5251,13 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `seed: optional number` @@ -4999,8 +5272,8 @@ Kicks off a new run for a given evaluation, specifying the data source, and what Configuration options for a text response from the model. Can be plain text or structured JSON data. Learn more: - - [Text inputs and outputs](/docs/guides/text) - - [Structured Outputs](/docs/guides/structured-outputs) + - [Text inputs and outputs](https://platform.openai.com/docs/guides/text) + - [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs) - `format: optional ResponseFormatTextConfig` @@ -5008,7 +5281,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what Configuring `{ "type": "json_schema" }` enables Structured Outputs, which ensures the model will match your supplied JSON schema. Learn more in the - [Structured Outputs guide](/docs/guides/structured-outputs). + [Structured Outputs guide](https://platform.openai.com/docs/guides/structured-outputs). The default format is `{ "type": "text" }` with no additional options. @@ -5025,7 +5298,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `ResponseFormatTextJSONSchemaConfig object { name, schema, type, 2 more }` JSON Schema response format. Used to generate structured JSON responses. - Learn more about [Structured Outputs](/docs/guides/structured-outputs). + Learn more about [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs). - `name: string` @@ -5054,7 +5327,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). - `ResponseFormatJSONObject object { type }` @@ -5063,7 +5336,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what model will not generate JSON without a system or user message instructing it to do so. - - `tools: optional array of object { name, parameters, strict, 3 more } or object { type, vector_store_ids, filters, 2 more } or object { type } or 12 more` + - `tools: optional array of object { name, parameters, strict, 5 more } or object { type, vector_store_ids, filters, 2 more } or object { type } or 13 more` An array of tools the model may call while generating a response. You can specify which tool to use by setting the `tool_choice` parameter. @@ -5071,14 +5344,14 @@ Kicks off a new run for a given evaluation, specifying the data source, and what The two categories of tools you can provide the model are: - **Built-in tools**: Tools that are provided by OpenAI that extend the - model's capabilities, like [web search](/docs/guides/tools-web-search) - or [file search](/docs/guides/tools-file-search). Learn more about - [built-in tools](/docs/guides/tools). + model's capabilities, like [web search](https://platform.openai.com/docs/guides/tools-web-search) + or [file search](https://platform.openai.com/docs/guides/tools-file-search). Learn more about + [built-in tools](https://platform.openai.com/docs/guides/tools). - **Function calls (custom tools)**: Functions that are defined by you, enabling the model to call your own code. Learn more about - [function calling](/docs/guides/function-calling). + [function calling](https://platform.openai.com/docs/guides/function-calling). - - `Function object { name, parameters, strict, 3 more }` + - `Function object { name, parameters, strict, 5 more }` Defines a function in your own code the model can choose to call. Learn more about [function calling](https://platform.openai.com/docs/guides/function-calling). @@ -5092,7 +5365,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `strict: boolean` - Whether to enforce strict parameter validation. Default `true`. + Whether strict parameter validation is enforced for this function tool. - `type: "function"` @@ -5100,6 +5373,14 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"function"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this function is deferred and loaded via tool search. @@ -5108,6 +5389,10 @@ Kicks off a new run for a given evaluation, specifying the data source, and what A description of the function. Used by the model to determine whether or not to call the function. + - `output_schema: optional map[unknown]` + + A JSON schema object describing the JSON value encoded in string outputs for this function. + - `FileSearch object { type, vector_store_ids, filters, 2 more }` A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](https://platform.openai.com/docs/guides/tools-file-search). @@ -5278,7 +5563,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `WebSearch object { type, filters, search_context_size, user_location }` Search the Internet for sources related to the prompt. Learn more about the - [web search tool](/docs/guides/tools-web-search). + [web search tool](https://platform.openai.com/docs/guides/tools-web-search). - `type: "web_search" or "web_search_2025_08_26"` @@ -5335,10 +5620,10 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"approximate"` - - `Mcp object { server_label, type, allowed_tools, 8 more }` + - `Mcp object { server_label, type, allowed_callers, 9 more }` Give the model access to additional tools via remote Model Context Protocol - (MCP) servers. [Learn more about MCP](/docs/guides/tools-remote-mcp). + (MCP) servers. [Learn more about MCP](https://platform.openai.com/docs/guides/tools-remote-mcp). - `server_label: string` @@ -5350,6 +5635,14 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"mcp"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `allowed_tools: optional array of string or object { read_only, tool_names }` List of allowed tool names or a filter object. @@ -5382,7 +5675,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what Identifier for service connectors, like those available in ChatGPT. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more - about service connectors [here](/docs/guides/tools-remote-mcp#connectors). + about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors). Currently supported `connector_id` values are: @@ -5482,7 +5775,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what The Secure MCP Tunnel ID to use instead of a direct server URL. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. - - `CodeInterpreter object { container, type }` + - `CodeInterpreter object { container, type, allowed_callers }` A tool that runs Python code to help generate a response to a prompt. @@ -5568,6 +5861,22 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"code_interpreter"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + + - `ProgrammaticToolCalling object { type }` + + - `type: "programmatic_tool_calling"` + + The type of the tool. Always `programmatic_tool_calling`. + + - `"programmatic_tool_calling"` + - `ImageGeneration object { type, action, background, 9 more }` A tool that generates images using the GPT image models. @@ -5590,8 +5899,19 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `background: optional "transparent" or "opaque" or "auto"` - Background type for the generated image. One of `transparent`, - `opaque`, or `auto`. Default: `auto`. + Allows to set transparency for the background of the generated image(s). + This parameter is only supported for GPT image models that support + transparent backgrounds. Must be one of `transparent`, `opaque`, or + `auto` (default value). When `auto` is used, the model will + automatically determine the best background for the image. + + `gpt-image-2` and `gpt-image-2-2026-04-21` do not support + transparent backgrounds. Requests with `background` set to + `transparent` will return an error for these models; use `opaque` or + `auto` instead. + + If `transparent`, the output format needs to support transparency, + so it should be set to either `png` (default value) or `webp`. - `"transparent"` @@ -5620,13 +5940,13 @@ Kicks off a new run for a given evaluation, specifying the data source, and what Base64-encoded mask image. - - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5"` + - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-2" or 3 more` The image generation model to use. Default: `gpt-image-1`. - `string` - - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5"` + - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-2" or 3 more` The image generation model to use. Default: `gpt-image-1`. @@ -5634,8 +5954,14 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"gpt-image-1-mini"` + - `"gpt-image-2"` + + - `"gpt-image-2-2026-04-21"` + - `"gpt-image-1.5"` + - `"chatgpt-image-latest"` + - `moderation: optional "auto" or "low"` Moderation level for the generated image. Default: `auto`. @@ -5704,7 +6030,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"local_shell"` - - `Shell object { type, environment }` + - `Shell object { type, allowed_callers, environment }` A tool that allows the model to execute shell commands. @@ -5714,6 +6040,14 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"shell"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `environment: optional ContainerAuto or LocalEnvironment or ContainerReference` - `ContainerAuto object { type, file_ids, memory_limit, 2 more }` @@ -5840,9 +6174,9 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"container_reference"` - - `Custom object { name, type, defer_loading, 2 more }` + - `Custom object { name, type, allowed_callers, 3 more }` - A custom tool that processes input using a specified format. Learn more about [custom tools](/docs/guides/function-calling#custom-tools) + A custom tool that processes input using a specified format. Learn more about [custom tools](https://platform.openai.com/docs/guides/function-calling#custom-tools) - `name: string` @@ -5854,6 +6188,14 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"custom"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this tool should be deferred and discovered via tool search. @@ -5910,11 +6252,11 @@ Kicks off a new run for a given evaluation, specifying the data source, and what The namespace name used in tool calls (for example, `crm`). - - `tools: array of object { name, type, defer_loading, 3 more } or object { name, type, defer_loading, 2 more }` + - `tools: array of object { name, type, allowed_callers, 5 more } or object { name, type, allowed_callers, 3 more }` The function/custom tools available inside this namespace. - - `Function object { name, type, defer_loading, 3 more }` + - `Function object { name, type, allowed_callers, 5 more }` - `name: string` @@ -5922,19 +6264,33 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"function"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this function should be deferred and discovered via tool search. - `description: optional string` + - `output_schema: optional map[unknown]` + + A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs. + - `parameters: optional unknown` - `strict: optional boolean` - - `Custom object { name, type, defer_loading, 2 more }` + Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise. - A custom tool that processes input using a specified format. Learn more about [custom tools](/docs/guides/function-calling#custom-tools) + - `Custom object { name, type, allowed_callers, 3 more }` + + A custom tool that processes input using a specified format. Learn more about [custom tools](https://platform.openai.com/docs/guides/function-calling#custom-tools) - `name: string` @@ -5946,6 +6302,14 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"custom"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this tool should be deferred and discovered via tool search. @@ -6044,7 +6408,7 @@ Kicks off a new run for a given evaluation, specifying the data source, and what The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`. - - `ApplyPatch object { type }` + - `ApplyPatch object { type, allowed_callers }` Allows the assistant to create, delete, or update files using unified diffs. @@ -6054,6 +6418,14 @@ Kicks off a new run for a given evaluation, specifying the data source, and what - `"apply_patch"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `top_p: optional number` An alternative to temperature for nucleus sampling; 1.0 includes all tokens. @@ -6334,7 +6706,7 @@ curl https://api.openai.com/v1/evals/eval_67e579652b548190aaa83ada4b125f47/runs **get** `/evals/{eval_id}/runs/{run_id}` -Get an evaluation run by ID. +Get an eval run ### Path Parameters @@ -6364,7 +6736,7 @@ Get an evaluation run by ID. Determines what populates the `item` namespace in the data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -6380,7 +6752,7 @@ Get an evaluation run by ID. - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -6406,7 +6778,7 @@ Get an evaluation run by ID. Determines what populates the `item` namespace in this run's data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -6422,7 +6794,7 @@ Get an evaluation run by ID. - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -6434,7 +6806,7 @@ Get an evaluation run by ID. - `"file_id"` - - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }` + - `StoredCompletions object { type, created_after, created_before, 3 more }` A StoredCompletionsRunDataSource configuration describing a set of filters @@ -6479,7 +6851,7 @@ Get an evaluation run by ID. Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - - `TemplateInputMessages object { template, type }` + - `Template object { template, type }` - `template: array of EasyInputMessage or object { content, role, type }` @@ -6507,7 +6879,7 @@ Get an evaluation run by ID. A list of one or many input items to the model, containing different content types. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -6521,9 +6893,19 @@ Get an evaluation run by ID. - `"input_text"` - - `ResponseInputImage object { detail, type, file_id, image_url }` + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. - An image input to the model. Learn about [image inputs](/docs/guides/vision). + - `"explicit"` + + - `ResponseInputImage object { detail, type, file_id, 2 more }` + + An image input to the model. Learn about [image inputs](https://platform.openai.com/docs/guides/vision). - `detail: "low" or "high" or "auto" or "original"` @@ -6551,7 +6933,17 @@ Get an evaluation run by ID. The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL. - - `ResponseInputFile object { type, detail, file_data, 3 more }` + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + + - `ResponseInputFile object { type, detail, file_data, 4 more }` A file input to the model. @@ -6561,9 +6953,11 @@ Get an evaluation run by ID. - `"input_file"` - - `detail: optional "low" or "high"` + - `detail: optional "auto" or "low" or "high"` + + The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`. - The detail level of the file to be sent to the model. Use `low` for the default rendering behavior, or `high` to render the file at higher quality. Defaults to `low`. + - `"auto"` - `"low"` @@ -6585,6 +6979,16 @@ Get an evaluation run by ID. The name of the file to be sent to the model. + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or @@ -6614,7 +7018,7 @@ Get an evaluation run by ID. - `"message"` - - `EvalMessageObject object { content, role, type }` + - `EvalItem object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take @@ -6630,7 +7034,7 @@ Get an evaluation run by ID. A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -6700,7 +7104,7 @@ Get an evaluation run by ID. A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -6765,7 +7169,7 @@ Get an evaluation run by ID. - `"template"` - - `ItemReferenceInputMessages object { item_reference, type }` + - `ItemReference object { item_reference, type }` - `item_reference: string` @@ -6789,16 +7193,13 @@ Get an evaluation run by ID. - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `"none"` @@ -6812,6 +7213,8 @@ Get an evaluation run by ID. - `"xhigh"` + - `"max"` + - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject` An object specifying the format that the model must output. @@ -6819,7 +7222,7 @@ Get an evaluation run by ID. Setting to `{ "type": "json_schema", "json_schema": {...} }` enables Structured Outputs which ensures the model will match your supplied JSON schema. Learn more in the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` @@ -6838,7 +7241,7 @@ Get an evaluation run by ID. - `ResponseFormatJSONSchema object { json_schema, type }` JSON Schema response format. Used to generate structured JSON responses. - Learn more about [Structured Outputs](/docs/guides/structured-outputs). + Learn more about [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs). - `json_schema: object { name, description, schema, strict }` @@ -6865,7 +7268,7 @@ Get an evaluation run by ID. If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). - `type: "json_schema"` @@ -6910,13 +7313,13 @@ Get an evaluation run by ID. - `parameters: optional FunctionParameters` - The parameters the functions accepts, described as a JSON Schema object. See the [guide](/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. + The parameters the functions accepts, described as a JSON Schema object. See the [guide](https://platform.openai.com/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. Omitting `parameters` defines a function with an empty parameter list. - `strict: optional boolean` - Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/docs/guides/function-calling). + Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](https://platform.openai.com/docs/guides/function-calling). - `type: "function"` @@ -6928,7 +7331,7 @@ Get an evaluation run by ID. An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `ResponsesRunDataSource object { source, type, input_messages, 2 more }` + - `Responses object { source, type, input_messages, 2 more }` A ResponsesRunDataSource object describing a model sampling configuration. @@ -6936,7 +7339,7 @@ Get an evaluation run by ID. Determines what populates the `item` namespace in this run's data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -6952,7 +7355,7 @@ Get an evaluation run by ID. - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -6964,7 +7367,7 @@ Get an evaluation run by ID. - `"file_id"` - - `EvalResponsesSource object { type, created_after, created_before, 8 more }` + - `Responses object { type, created_after, created_before, 8 more }` A EvalResponsesSource object describing a run data source configuration. @@ -6996,16 +7399,13 @@ Get an evaluation run by ID. - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `temperature: optional number` @@ -7033,7 +7433,7 @@ Get an evaluation run by ID. Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - - `InputMessagesTemplate object { template, type }` + - `Template object { template, type }` - `template: array of object { content, role } or object { content, role, type }` @@ -7049,7 +7449,7 @@ Get an evaluation run by ID. The role of the message (e.g. "system", "assistant", "user"). - - `EvalMessageObject object { content, role, type }` + - `EvalItem object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take @@ -7065,7 +7465,7 @@ Get an evaluation run by ID. A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -7135,7 +7535,7 @@ Get an evaluation run by ID. - `"template"` - - `InputMessagesItemReference object { item_reference, type }` + - `ItemReference object { item_reference, type }` - `item_reference: string` @@ -7159,16 +7559,13 @@ Get an evaluation run by ID. - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `seed: optional number` @@ -7183,8 +7580,8 @@ Get an evaluation run by ID. Configuration options for a text response from the model. Can be plain text or structured JSON data. Learn more: - - [Text inputs and outputs](/docs/guides/text) - - [Structured Outputs](/docs/guides/structured-outputs) + - [Text inputs and outputs](https://platform.openai.com/docs/guides/text) + - [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs) - `format: optional ResponseFormatTextConfig` @@ -7192,7 +7589,7 @@ Get an evaluation run by ID. Configuring `{ "type": "json_schema" }` enables Structured Outputs, which ensures the model will match your supplied JSON schema. Learn more in the - [Structured Outputs guide](/docs/guides/structured-outputs). + [Structured Outputs guide](https://platform.openai.com/docs/guides/structured-outputs). The default format is `{ "type": "text" }` with no additional options. @@ -7209,7 +7606,7 @@ Get an evaluation run by ID. - `ResponseFormatTextJSONSchemaConfig object { name, schema, type, 2 more }` JSON Schema response format. Used to generate structured JSON responses. - Learn more about [Structured Outputs](/docs/guides/structured-outputs). + Learn more about [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs). - `name: string` @@ -7238,7 +7635,7 @@ Get an evaluation run by ID. If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). - `ResponseFormatJSONObject object { type }` @@ -7247,7 +7644,7 @@ Get an evaluation run by ID. model will not generate JSON without a system or user message instructing it to do so. - - `tools: optional array of object { name, parameters, strict, 3 more } or object { type, vector_store_ids, filters, 2 more } or object { type } or 12 more` + - `tools: optional array of object { name, parameters, strict, 5 more } or object { type, vector_store_ids, filters, 2 more } or object { type } or 13 more` An array of tools the model may call while generating a response. You can specify which tool to use by setting the `tool_choice` parameter. @@ -7255,14 +7652,14 @@ Get an evaluation run by ID. The two categories of tools you can provide the model are: - **Built-in tools**: Tools that are provided by OpenAI that extend the - model's capabilities, like [web search](/docs/guides/tools-web-search) - or [file search](/docs/guides/tools-file-search). Learn more about - [built-in tools](/docs/guides/tools). + model's capabilities, like [web search](https://platform.openai.com/docs/guides/tools-web-search) + or [file search](https://platform.openai.com/docs/guides/tools-file-search). Learn more about + [built-in tools](https://platform.openai.com/docs/guides/tools). - **Function calls (custom tools)**: Functions that are defined by you, enabling the model to call your own code. Learn more about - [function calling](/docs/guides/function-calling). + [function calling](https://platform.openai.com/docs/guides/function-calling). - - `Function object { name, parameters, strict, 3 more }` + - `Function object { name, parameters, strict, 5 more }` Defines a function in your own code the model can choose to call. Learn more about [function calling](https://platform.openai.com/docs/guides/function-calling). @@ -7276,7 +7673,7 @@ Get an evaluation run by ID. - `strict: boolean` - Whether to enforce strict parameter validation. Default `true`. + Whether strict parameter validation is enforced for this function tool. - `type: "function"` @@ -7284,6 +7681,14 @@ Get an evaluation run by ID. - `"function"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this function is deferred and loaded via tool search. @@ -7292,6 +7697,10 @@ Get an evaluation run by ID. A description of the function. Used by the model to determine whether or not to call the function. + - `output_schema: optional map[unknown]` + + A JSON schema object describing the JSON value encoded in string outputs for this function. + - `FileSearch object { type, vector_store_ids, filters, 2 more }` A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](https://platform.openai.com/docs/guides/tools-file-search). @@ -7462,7 +7871,7 @@ Get an evaluation run by ID. - `WebSearch object { type, filters, search_context_size, user_location }` Search the Internet for sources related to the prompt. Learn more about the - [web search tool](/docs/guides/tools-web-search). + [web search tool](https://platform.openai.com/docs/guides/tools-web-search). - `type: "web_search" or "web_search_2025_08_26"` @@ -7519,10 +7928,10 @@ Get an evaluation run by ID. - `"approximate"` - - `Mcp object { server_label, type, allowed_tools, 8 more }` + - `Mcp object { server_label, type, allowed_callers, 9 more }` Give the model access to additional tools via remote Model Context Protocol - (MCP) servers. [Learn more about MCP](/docs/guides/tools-remote-mcp). + (MCP) servers. [Learn more about MCP](https://platform.openai.com/docs/guides/tools-remote-mcp). - `server_label: string` @@ -7534,6 +7943,14 @@ Get an evaluation run by ID. - `"mcp"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `allowed_tools: optional array of string or object { read_only, tool_names }` List of allowed tool names or a filter object. @@ -7566,7 +7983,7 @@ Get an evaluation run by ID. Identifier for service connectors, like those available in ChatGPT. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more - about service connectors [here](/docs/guides/tools-remote-mcp#connectors). + about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors). Currently supported `connector_id` values are: @@ -7666,7 +8083,7 @@ Get an evaluation run by ID. The Secure MCP Tunnel ID to use instead of a direct server URL. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. - - `CodeInterpreter object { container, type }` + - `CodeInterpreter object { container, type, allowed_callers }` A tool that runs Python code to help generate a response to a prompt. @@ -7752,6 +8169,22 @@ Get an evaluation run by ID. - `"code_interpreter"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + + - `ProgrammaticToolCalling object { type }` + + - `type: "programmatic_tool_calling"` + + The type of the tool. Always `programmatic_tool_calling`. + + - `"programmatic_tool_calling"` + - `ImageGeneration object { type, action, background, 9 more }` A tool that generates images using the GPT image models. @@ -7774,8 +8207,19 @@ Get an evaluation run by ID. - `background: optional "transparent" or "opaque" or "auto"` - Background type for the generated image. One of `transparent`, - `opaque`, or `auto`. Default: `auto`. + Allows to set transparency for the background of the generated image(s). + This parameter is only supported for GPT image models that support + transparent backgrounds. Must be one of `transparent`, `opaque`, or + `auto` (default value). When `auto` is used, the model will + automatically determine the best background for the image. + + `gpt-image-2` and `gpt-image-2-2026-04-21` do not support + transparent backgrounds. Requests with `background` set to + `transparent` will return an error for these models; use `opaque` or + `auto` instead. + + If `transparent`, the output format needs to support transparency, + so it should be set to either `png` (default value) or `webp`. - `"transparent"` @@ -7804,13 +8248,13 @@ Get an evaluation run by ID. Base64-encoded mask image. - - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5"` + - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-2" or 3 more` The image generation model to use. Default: `gpt-image-1`. - `string` - - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5"` + - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-2" or 3 more` The image generation model to use. Default: `gpt-image-1`. @@ -7818,8 +8262,14 @@ Get an evaluation run by ID. - `"gpt-image-1-mini"` + - `"gpt-image-2"` + + - `"gpt-image-2-2026-04-21"` + - `"gpt-image-1.5"` + - `"chatgpt-image-latest"` + - `moderation: optional "auto" or "low"` Moderation level for the generated image. Default: `auto`. @@ -7888,7 +8338,7 @@ Get an evaluation run by ID. - `"local_shell"` - - `Shell object { type, environment }` + - `Shell object { type, allowed_callers, environment }` A tool that allows the model to execute shell commands. @@ -7898,6 +8348,14 @@ Get an evaluation run by ID. - `"shell"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `environment: optional ContainerAuto or LocalEnvironment or ContainerReference` - `ContainerAuto object { type, file_ids, memory_limit, 2 more }` @@ -8024,9 +8482,9 @@ Get an evaluation run by ID. - `"container_reference"` - - `Custom object { name, type, defer_loading, 2 more }` + - `Custom object { name, type, allowed_callers, 3 more }` - A custom tool that processes input using a specified format. Learn more about [custom tools](/docs/guides/function-calling#custom-tools) + A custom tool that processes input using a specified format. Learn more about [custom tools](https://platform.openai.com/docs/guides/function-calling#custom-tools) - `name: string` @@ -8038,6 +8496,14 @@ Get an evaluation run by ID. - `"custom"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this tool should be deferred and discovered via tool search. @@ -8094,11 +8560,11 @@ Get an evaluation run by ID. The namespace name used in tool calls (for example, `crm`). - - `tools: array of object { name, type, defer_loading, 3 more } or object { name, type, defer_loading, 2 more }` + - `tools: array of object { name, type, allowed_callers, 5 more } or object { name, type, allowed_callers, 3 more }` The function/custom tools available inside this namespace. - - `Function object { name, type, defer_loading, 3 more }` + - `Function object { name, type, allowed_callers, 5 more }` - `name: string` @@ -8106,19 +8572,33 @@ Get an evaluation run by ID. - `"function"` - - `defer_loading: optional boolean` + - `allowed_callers: optional array of "direct" or "programmatic"` - Whether this function should be deferred and discovered via tool search. + The tool invocation context(s). - - `description: optional string` + - `"direct"` + + - `"programmatic"` + + - `defer_loading: optional boolean` + + Whether this function should be deferred and discovered via tool search. + + - `description: optional string` + + - `output_schema: optional map[unknown]` + + A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs. - `parameters: optional unknown` - `strict: optional boolean` - - `Custom object { name, type, defer_loading, 2 more }` + Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise. - A custom tool that processes input using a specified format. Learn more about [custom tools](/docs/guides/function-calling#custom-tools) + - `Custom object { name, type, allowed_callers, 3 more }` + + A custom tool that processes input using a specified format. Learn more about [custom tools](https://platform.openai.com/docs/guides/function-calling#custom-tools) - `name: string` @@ -8130,6 +8610,14 @@ Get an evaluation run by ID. - `"custom"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this tool should be deferred and discovered via tool search. @@ -8228,7 +8716,7 @@ Get an evaluation run by ID. The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`. - - `ApplyPatch object { type }` + - `ApplyPatch object { type, allowed_callers }` Allows the assistant to create, delete, or update files using unified diffs. @@ -8238,6 +8726,14 @@ Get an evaluation run by ID. - `"apply_patch"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `top_p: optional number` An alternative to temperature for nucleus sampling; 1.0 includes all tokens. @@ -8584,7 +9080,7 @@ curl https://api.openai.com/v1/evals/eval_67abd54d9b0081909a86353f6fb9317a/runs/ **post** `/evals/{eval_id}/runs/{run_id}` -Cancel an ongoing evaluation run. +Cancel eval run ### Path Parameters @@ -8614,7 +9110,7 @@ Cancel an ongoing evaluation run. Determines what populates the `item` namespace in the data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -8630,7 +9126,7 @@ Cancel an ongoing evaluation run. - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -8656,7 +9152,7 @@ Cancel an ongoing evaluation run. Determines what populates the `item` namespace in this run's data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -8672,7 +9168,7 @@ Cancel an ongoing evaluation run. - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -8684,7 +9180,7 @@ Cancel an ongoing evaluation run. - `"file_id"` - - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }` + - `StoredCompletions object { type, created_after, created_before, 3 more }` A StoredCompletionsRunDataSource configuration describing a set of filters @@ -8729,7 +9225,7 @@ Cancel an ongoing evaluation run. Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - - `TemplateInputMessages object { template, type }` + - `Template object { template, type }` - `template: array of EasyInputMessage or object { content, role, type }` @@ -8757,7 +9253,7 @@ Cancel an ongoing evaluation run. A list of one or many input items to the model, containing different content types. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -8771,9 +9267,19 @@ Cancel an ongoing evaluation run. - `"input_text"` - - `ResponseInputImage object { detail, type, file_id, image_url }` + - `prompt_cache_breakpoint: optional object { mode }` - An image input to the model. Learn about [image inputs](/docs/guides/vision). + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + + - `ResponseInputImage object { detail, type, file_id, 2 more }` + + An image input to the model. Learn about [image inputs](https://platform.openai.com/docs/guides/vision). - `detail: "low" or "high" or "auto" or "original"` @@ -8801,7 +9307,17 @@ Cancel an ongoing evaluation run. The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL. - - `ResponseInputFile object { type, detail, file_data, 3 more }` + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + + - `ResponseInputFile object { type, detail, file_data, 4 more }` A file input to the model. @@ -8811,9 +9327,11 @@ Cancel an ongoing evaluation run. - `"input_file"` - - `detail: optional "low" or "high"` + - `detail: optional "auto" or "low" or "high"` + + The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`. - The detail level of the file to be sent to the model. Use `low` for the default rendering behavior, or `high` to render the file at higher quality. Defaults to `low`. + - `"auto"` - `"low"` @@ -8835,6 +9353,16 @@ Cancel an ongoing evaluation run. The name of the file to be sent to the model. + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or @@ -8864,7 +9392,7 @@ Cancel an ongoing evaluation run. - `"message"` - - `EvalMessageObject object { content, role, type }` + - `EvalItem object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take @@ -8880,7 +9408,7 @@ Cancel an ongoing evaluation run. A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -8950,7 +9478,7 @@ Cancel an ongoing evaluation run. A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -9015,7 +9543,7 @@ Cancel an ongoing evaluation run. - `"template"` - - `ItemReferenceInputMessages object { item_reference, type }` + - `ItemReference object { item_reference, type }` - `item_reference: string` @@ -9039,16 +9567,13 @@ Cancel an ongoing evaluation run. - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `"none"` @@ -9062,6 +9587,8 @@ Cancel an ongoing evaluation run. - `"xhigh"` + - `"max"` + - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject` An object specifying the format that the model must output. @@ -9069,7 +9596,7 @@ Cancel an ongoing evaluation run. Setting to `{ "type": "json_schema", "json_schema": {...} }` enables Structured Outputs which ensures the model will match your supplied JSON schema. Learn more in the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` @@ -9088,7 +9615,7 @@ Cancel an ongoing evaluation run. - `ResponseFormatJSONSchema object { json_schema, type }` JSON Schema response format. Used to generate structured JSON responses. - Learn more about [Structured Outputs](/docs/guides/structured-outputs). + Learn more about [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs). - `json_schema: object { name, description, schema, strict }` @@ -9115,7 +9642,7 @@ Cancel an ongoing evaluation run. If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). - `type: "json_schema"` @@ -9160,13 +9687,13 @@ Cancel an ongoing evaluation run. - `parameters: optional FunctionParameters` - The parameters the functions accepts, described as a JSON Schema object. See the [guide](/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. + The parameters the functions accepts, described as a JSON Schema object. See the [guide](https://platform.openai.com/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. Omitting `parameters` defines a function with an empty parameter list. - `strict: optional boolean` - Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/docs/guides/function-calling). + Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](https://platform.openai.com/docs/guides/function-calling). - `type: "function"` @@ -9178,7 +9705,7 @@ Cancel an ongoing evaluation run. An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `ResponsesRunDataSource object { source, type, input_messages, 2 more }` + - `Responses object { source, type, input_messages, 2 more }` A ResponsesRunDataSource object describing a model sampling configuration. @@ -9186,7 +9713,7 @@ Cancel an ongoing evaluation run. Determines what populates the `item` namespace in this run's data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -9202,7 +9729,7 @@ Cancel an ongoing evaluation run. - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -9214,7 +9741,7 @@ Cancel an ongoing evaluation run. - `"file_id"` - - `EvalResponsesSource object { type, created_after, created_before, 8 more }` + - `Responses object { type, created_after, created_before, 8 more }` A EvalResponsesSource object describing a run data source configuration. @@ -9246,16 +9773,13 @@ Cancel an ongoing evaluation run. - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `temperature: optional number` @@ -9283,7 +9807,7 @@ Cancel an ongoing evaluation run. Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - - `InputMessagesTemplate object { template, type }` + - `Template object { template, type }` - `template: array of object { content, role } or object { content, role, type }` @@ -9299,7 +9823,7 @@ Cancel an ongoing evaluation run. The role of the message (e.g. "system", "assistant", "user"). - - `EvalMessageObject object { content, role, type }` + - `EvalItem object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take @@ -9315,7 +9839,7 @@ Cancel an ongoing evaluation run. A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -9385,7 +9909,7 @@ Cancel an ongoing evaluation run. - `"template"` - - `InputMessagesItemReference object { item_reference, type }` + - `ItemReference object { item_reference, type }` - `item_reference: string` @@ -9409,16 +9933,13 @@ Cancel an ongoing evaluation run. - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `seed: optional number` @@ -9433,8 +9954,8 @@ Cancel an ongoing evaluation run. Configuration options for a text response from the model. Can be plain text or structured JSON data. Learn more: - - [Text inputs and outputs](/docs/guides/text) - - [Structured Outputs](/docs/guides/structured-outputs) + - [Text inputs and outputs](https://platform.openai.com/docs/guides/text) + - [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs) - `format: optional ResponseFormatTextConfig` @@ -9442,7 +9963,7 @@ Cancel an ongoing evaluation run. Configuring `{ "type": "json_schema" }` enables Structured Outputs, which ensures the model will match your supplied JSON schema. Learn more in the - [Structured Outputs guide](/docs/guides/structured-outputs). + [Structured Outputs guide](https://platform.openai.com/docs/guides/structured-outputs). The default format is `{ "type": "text" }` with no additional options. @@ -9459,7 +9980,7 @@ Cancel an ongoing evaluation run. - `ResponseFormatTextJSONSchemaConfig object { name, schema, type, 2 more }` JSON Schema response format. Used to generate structured JSON responses. - Learn more about [Structured Outputs](/docs/guides/structured-outputs). + Learn more about [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs). - `name: string` @@ -9488,7 +10009,7 @@ Cancel an ongoing evaluation run. If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). - `ResponseFormatJSONObject object { type }` @@ -9497,7 +10018,7 @@ Cancel an ongoing evaluation run. model will not generate JSON without a system or user message instructing it to do so. - - `tools: optional array of object { name, parameters, strict, 3 more } or object { type, vector_store_ids, filters, 2 more } or object { type } or 12 more` + - `tools: optional array of object { name, parameters, strict, 5 more } or object { type, vector_store_ids, filters, 2 more } or object { type } or 13 more` An array of tools the model may call while generating a response. You can specify which tool to use by setting the `tool_choice` parameter. @@ -9505,14 +10026,14 @@ Cancel an ongoing evaluation run. The two categories of tools you can provide the model are: - **Built-in tools**: Tools that are provided by OpenAI that extend the - model's capabilities, like [web search](/docs/guides/tools-web-search) - or [file search](/docs/guides/tools-file-search). Learn more about - [built-in tools](/docs/guides/tools). + model's capabilities, like [web search](https://platform.openai.com/docs/guides/tools-web-search) + or [file search](https://platform.openai.com/docs/guides/tools-file-search). Learn more about + [built-in tools](https://platform.openai.com/docs/guides/tools). - **Function calls (custom tools)**: Functions that are defined by you, enabling the model to call your own code. Learn more about - [function calling](/docs/guides/function-calling). + [function calling](https://platform.openai.com/docs/guides/function-calling). - - `Function object { name, parameters, strict, 3 more }` + - `Function object { name, parameters, strict, 5 more }` Defines a function in your own code the model can choose to call. Learn more about [function calling](https://platform.openai.com/docs/guides/function-calling). @@ -9526,7 +10047,7 @@ Cancel an ongoing evaluation run. - `strict: boolean` - Whether to enforce strict parameter validation. Default `true`. + Whether strict parameter validation is enforced for this function tool. - `type: "function"` @@ -9534,6 +10055,14 @@ Cancel an ongoing evaluation run. - `"function"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this function is deferred and loaded via tool search. @@ -9542,6 +10071,10 @@ Cancel an ongoing evaluation run. A description of the function. Used by the model to determine whether or not to call the function. + - `output_schema: optional map[unknown]` + + A JSON schema object describing the JSON value encoded in string outputs for this function. + - `FileSearch object { type, vector_store_ids, filters, 2 more }` A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](https://platform.openai.com/docs/guides/tools-file-search). @@ -9712,7 +10245,7 @@ Cancel an ongoing evaluation run. - `WebSearch object { type, filters, search_context_size, user_location }` Search the Internet for sources related to the prompt. Learn more about the - [web search tool](/docs/guides/tools-web-search). + [web search tool](https://platform.openai.com/docs/guides/tools-web-search). - `type: "web_search" or "web_search_2025_08_26"` @@ -9769,10 +10302,10 @@ Cancel an ongoing evaluation run. - `"approximate"` - - `Mcp object { server_label, type, allowed_tools, 8 more }` + - `Mcp object { server_label, type, allowed_callers, 9 more }` Give the model access to additional tools via remote Model Context Protocol - (MCP) servers. [Learn more about MCP](/docs/guides/tools-remote-mcp). + (MCP) servers. [Learn more about MCP](https://platform.openai.com/docs/guides/tools-remote-mcp). - `server_label: string` @@ -9784,6 +10317,14 @@ Cancel an ongoing evaluation run. - `"mcp"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `allowed_tools: optional array of string or object { read_only, tool_names }` List of allowed tool names or a filter object. @@ -9816,7 +10357,7 @@ Cancel an ongoing evaluation run. Identifier for service connectors, like those available in ChatGPT. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more - about service connectors [here](/docs/guides/tools-remote-mcp#connectors). + about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors). Currently supported `connector_id` values are: @@ -9916,7 +10457,7 @@ Cancel an ongoing evaluation run. The Secure MCP Tunnel ID to use instead of a direct server URL. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. - - `CodeInterpreter object { container, type }` + - `CodeInterpreter object { container, type, allowed_callers }` A tool that runs Python code to help generate a response to a prompt. @@ -10002,6 +10543,22 @@ Cancel an ongoing evaluation run. - `"code_interpreter"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + + - `ProgrammaticToolCalling object { type }` + + - `type: "programmatic_tool_calling"` + + The type of the tool. Always `programmatic_tool_calling`. + + - `"programmatic_tool_calling"` + - `ImageGeneration object { type, action, background, 9 more }` A tool that generates images using the GPT image models. @@ -10024,8 +10581,19 @@ Cancel an ongoing evaluation run. - `background: optional "transparent" or "opaque" or "auto"` - Background type for the generated image. One of `transparent`, - `opaque`, or `auto`. Default: `auto`. + Allows to set transparency for the background of the generated image(s). + This parameter is only supported for GPT image models that support + transparent backgrounds. Must be one of `transparent`, `opaque`, or + `auto` (default value). When `auto` is used, the model will + automatically determine the best background for the image. + + `gpt-image-2` and `gpt-image-2-2026-04-21` do not support + transparent backgrounds. Requests with `background` set to + `transparent` will return an error for these models; use `opaque` or + `auto` instead. + + If `transparent`, the output format needs to support transparency, + so it should be set to either `png` (default value) or `webp`. - `"transparent"` @@ -10054,13 +10622,13 @@ Cancel an ongoing evaluation run. Base64-encoded mask image. - - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5"` + - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-2" or 3 more` The image generation model to use. Default: `gpt-image-1`. - `string` - - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5"` + - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-2" or 3 more` The image generation model to use. Default: `gpt-image-1`. @@ -10068,8 +10636,14 @@ Cancel an ongoing evaluation run. - `"gpt-image-1-mini"` + - `"gpt-image-2"` + + - `"gpt-image-2-2026-04-21"` + - `"gpt-image-1.5"` + - `"chatgpt-image-latest"` + - `moderation: optional "auto" or "low"` Moderation level for the generated image. Default: `auto`. @@ -10138,7 +10712,7 @@ Cancel an ongoing evaluation run. - `"local_shell"` - - `Shell object { type, environment }` + - `Shell object { type, allowed_callers, environment }` A tool that allows the model to execute shell commands. @@ -10148,6 +10722,14 @@ Cancel an ongoing evaluation run. - `"shell"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `environment: optional ContainerAuto or LocalEnvironment or ContainerReference` - `ContainerAuto object { type, file_ids, memory_limit, 2 more }` @@ -10274,9 +10856,9 @@ Cancel an ongoing evaluation run. - `"container_reference"` - - `Custom object { name, type, defer_loading, 2 more }` + - `Custom object { name, type, allowed_callers, 3 more }` - A custom tool that processes input using a specified format. Learn more about [custom tools](/docs/guides/function-calling#custom-tools) + A custom tool that processes input using a specified format. Learn more about [custom tools](https://platform.openai.com/docs/guides/function-calling#custom-tools) - `name: string` @@ -10288,6 +10870,14 @@ Cancel an ongoing evaluation run. - `"custom"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this tool should be deferred and discovered via tool search. @@ -10344,11 +10934,11 @@ Cancel an ongoing evaluation run. The namespace name used in tool calls (for example, `crm`). - - `tools: array of object { name, type, defer_loading, 3 more } or object { name, type, defer_loading, 2 more }` + - `tools: array of object { name, type, allowed_callers, 5 more } or object { name, type, allowed_callers, 3 more }` The function/custom tools available inside this namespace. - - `Function object { name, type, defer_loading, 3 more }` + - `Function object { name, type, allowed_callers, 5 more }` - `name: string` @@ -10356,19 +10946,33 @@ Cancel an ongoing evaluation run. - `"function"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this function should be deferred and discovered via tool search. - `description: optional string` + - `output_schema: optional map[unknown]` + + A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs. + - `parameters: optional unknown` - `strict: optional boolean` - - `Custom object { name, type, defer_loading, 2 more }` + Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise. - A custom tool that processes input using a specified format. Learn more about [custom tools](/docs/guides/function-calling#custom-tools) + - `Custom object { name, type, allowed_callers, 3 more }` + + A custom tool that processes input using a specified format. Learn more about [custom tools](https://platform.openai.com/docs/guides/function-calling#custom-tools) - `name: string` @@ -10380,6 +10984,14 @@ Cancel an ongoing evaluation run. - `"custom"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this tool should be deferred and discovered via tool search. @@ -10478,7 +11090,7 @@ Cancel an ongoing evaluation run. The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`. - - `ApplyPatch object { type }` + - `ApplyPatch object { type, allowed_callers }` Allows the assistant to create, delete, or update files using unified diffs. @@ -10488,6 +11100,14 @@ Cancel an ongoing evaluation run. - `"apply_patch"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `top_p: optional number` An alternative to temperature for nucleus sampling; 1.0 includes all tokens. @@ -10836,7 +11456,7 @@ curl https://api.openai.com/v1/evals/eval_67abd54d9b0081909a86353f6fb9317a/runs/ **delete** `/evals/{eval_id}/runs/{run_id}` -Delete an eval run. +Delete eval run ### Path Parameters @@ -10901,7 +11521,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Determines what populates the `item` namespace in this run's data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -10917,7 +11537,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -10929,7 +11549,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_id"` - - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }` + - `StoredCompletions object { type, created_after, created_before, 3 more }` A StoredCompletionsRunDataSource configuration describing a set of filters @@ -10974,7 +11594,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - - `TemplateInputMessages object { template, type }` + - `Template object { template, type }` - `template: array of EasyInputMessage or object { content, role, type }` @@ -11002,7 +11622,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A list of one or many input items to the model, containing different content types. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -11016,9 +11636,19 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"input_text"` - - `ResponseInputImage object { detail, type, file_id, image_url }` + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. - An image input to the model. Learn about [image inputs](/docs/guides/vision). + - `"explicit"` + + - `ResponseInputImage object { detail, type, file_id, 2 more }` + + An image input to the model. Learn about [image inputs](https://platform.openai.com/docs/guides/vision). - `detail: "low" or "high" or "auto" or "original"` @@ -11046,7 +11676,17 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL. - - `ResponseInputFile object { type, detail, file_data, 3 more }` + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + + - `ResponseInputFile object { type, detail, file_data, 4 more }` A file input to the model. @@ -11056,9 +11696,11 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"input_file"` - - `detail: optional "low" or "high"` + - `detail: optional "auto" or "low" or "high"` + + The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`. - The detail level of the file to be sent to the model. Use `low` for the default rendering behavior, or `high` to render the file at higher quality. Defaults to `low`. + - `"auto"` - `"low"` @@ -11080,6 +11722,16 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The name of the file to be sent to the model. + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or @@ -11109,7 +11761,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"message"` - - `EvalMessageObject object { content, role, type }` + - `EvalItem object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take @@ -11125,7 +11777,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -11195,7 +11847,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -11260,7 +11912,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"template"` - - `ItemReferenceInputMessages object { item_reference, type }` + - `ItemReference object { item_reference, type }` - `item_reference: string` @@ -11284,16 +11936,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `"none"` @@ -11307,6 +11956,8 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"xhigh"` + - `"max"` + - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject` An object specifying the format that the model must output. @@ -11314,7 +11965,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Setting to `{ "type": "json_schema", "json_schema": {...} }` enables Structured Outputs which ensures the model will match your supplied JSON schema. Learn more in the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` @@ -11333,7 +11984,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `ResponseFormatJSONSchema object { json_schema, type }` JSON Schema response format. Used to generate structured JSON responses. - Learn more about [Structured Outputs](/docs/guides/structured-outputs). + Learn more about [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs). - `json_schema: object { name, description, schema, strict }` @@ -11360,7 +12011,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). - `type: "json_schema"` @@ -11405,13 +12056,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `parameters: optional FunctionParameters` - The parameters the functions accepts, described as a JSON Schema object. See the [guide](/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. + The parameters the functions accepts, described as a JSON Schema object. See the [guide](https://platform.openai.com/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. Omitting `parameters` defines a function with an empty parameter list. - `strict: optional boolean` - Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/docs/guides/function-calling). + Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](https://platform.openai.com/docs/guides/function-calling). - `type: "function"` @@ -11433,7 +12084,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Determines what populates the `item` namespace in the data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -11449,7 +12100,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -11507,7 +12158,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Determines what populates the `item` namespace in the data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -11523,7 +12174,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -11549,7 +12200,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Determines what populates the `item` namespace in this run's data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -11565,7 +12216,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -11577,7 +12228,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_id"` - - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }` + - `StoredCompletions object { type, created_after, created_before, 3 more }` A StoredCompletionsRunDataSource configuration describing a set of filters @@ -11622,7 +12273,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - - `TemplateInputMessages object { template, type }` + - `Template object { template, type }` - `template: array of EasyInputMessage or object { content, role, type }` @@ -11650,7 +12301,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A list of one or many input items to the model, containing different content types. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -11664,9 +12315,19 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"input_text"` - - `ResponseInputImage object { detail, type, file_id, image_url }` + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. - An image input to the model. Learn about [image inputs](/docs/guides/vision). + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + + - `ResponseInputImage object { detail, type, file_id, 2 more }` + + An image input to the model. Learn about [image inputs](https://platform.openai.com/docs/guides/vision). - `detail: "low" or "high" or "auto" or "original"` @@ -11694,7 +12355,17 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL. - - `ResponseInputFile object { type, detail, file_data, 3 more }` + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + + - `ResponseInputFile object { type, detail, file_data, 4 more }` A file input to the model. @@ -11704,9 +12375,11 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"input_file"` - - `detail: optional "low" or "high"` + - `detail: optional "auto" or "low" or "high"` - The detail level of the file to be sent to the model. Use `low` for the default rendering behavior, or `high` to render the file at higher quality. Defaults to `low`. + The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`. + + - `"auto"` - `"low"` @@ -11728,6 +12401,16 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The name of the file to be sent to the model. + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or @@ -11757,7 +12440,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"message"` - - `EvalMessageObject object { content, role, type }` + - `EvalItem object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take @@ -11773,7 +12456,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -11843,7 +12526,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -11908,7 +12591,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"template"` - - `ItemReferenceInputMessages object { item_reference, type }` + - `ItemReference object { item_reference, type }` - `item_reference: string` @@ -11932,16 +12615,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `"none"` @@ -11955,6 +12635,8 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"xhigh"` + - `"max"` + - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject` An object specifying the format that the model must output. @@ -11962,7 +12644,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Setting to `{ "type": "json_schema", "json_schema": {...} }` enables Structured Outputs which ensures the model will match your supplied JSON schema. Learn more in the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` @@ -11981,7 +12663,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `ResponseFormatJSONSchema object { json_schema, type }` JSON Schema response format. Used to generate structured JSON responses. - Learn more about [Structured Outputs](/docs/guides/structured-outputs). + Learn more about [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs). - `json_schema: object { name, description, schema, strict }` @@ -12008,7 +12690,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). - `type: "json_schema"` @@ -12053,13 +12735,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `parameters: optional FunctionParameters` - The parameters the functions accepts, described as a JSON Schema object. See the [guide](/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. + The parameters the functions accepts, described as a JSON Schema object. See the [guide](https://platform.openai.com/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. Omitting `parameters` defines a function with an empty parameter list. - `strict: optional boolean` - Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/docs/guides/function-calling). + Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](https://platform.openai.com/docs/guides/function-calling). - `type: "function"` @@ -12071,7 +12753,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `ResponsesRunDataSource object { source, type, input_messages, 2 more }` + - `Responses object { source, type, input_messages, 2 more }` A ResponsesRunDataSource object describing a model sampling configuration. @@ -12079,7 +12761,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Determines what populates the `item` namespace in this run's data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -12095,7 +12777,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -12107,7 +12789,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_id"` - - `EvalResponsesSource object { type, created_after, created_before, 8 more }` + - `Responses object { type, created_after, created_before, 8 more }` A EvalResponsesSource object describing a run data source configuration. @@ -12139,16 +12821,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `temperature: optional number` @@ -12176,7 +12855,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - - `InputMessagesTemplate object { template, type }` + - `Template object { template, type }` - `template: array of object { content, role } or object { content, role, type }` @@ -12192,7 +12871,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The role of the message (e.g. "system", "assistant", "user"). - - `EvalMessageObject object { content, role, type }` + - `EvalItem object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take @@ -12208,7 +12887,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -12278,7 +12957,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"template"` - - `InputMessagesItemReference object { item_reference, type }` + - `ItemReference object { item_reference, type }` - `item_reference: string` @@ -12302,16 +12981,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `seed: optional number` @@ -12326,8 +13002,8 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Configuration options for a text response from the model. Can be plain text or structured JSON data. Learn more: - - [Text inputs and outputs](/docs/guides/text) - - [Structured Outputs](/docs/guides/structured-outputs) + - [Text inputs and outputs](https://platform.openai.com/docs/guides/text) + - [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs) - `format: optional ResponseFormatTextConfig` @@ -12335,7 +13011,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Configuring `{ "type": "json_schema" }` enables Structured Outputs, which ensures the model will match your supplied JSON schema. Learn more in the - [Structured Outputs guide](/docs/guides/structured-outputs). + [Structured Outputs guide](https://platform.openai.com/docs/guides/structured-outputs). The default format is `{ "type": "text" }` with no additional options. @@ -12352,7 +13028,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `ResponseFormatTextJSONSchemaConfig object { name, schema, type, 2 more }` JSON Schema response format. Used to generate structured JSON responses. - Learn more about [Structured Outputs](/docs/guides/structured-outputs). + Learn more about [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs). - `name: string` @@ -12381,7 +13057,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). - `ResponseFormatJSONObject object { type }` @@ -12390,7 +13066,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ model will not generate JSON without a system or user message instructing it to do so. - - `tools: optional array of object { name, parameters, strict, 3 more } or object { type, vector_store_ids, filters, 2 more } or object { type } or 12 more` + - `tools: optional array of object { name, parameters, strict, 5 more } or object { type, vector_store_ids, filters, 2 more } or object { type } or 13 more` An array of tools the model may call while generating a response. You can specify which tool to use by setting the `tool_choice` parameter. @@ -12398,14 +13074,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The two categories of tools you can provide the model are: - **Built-in tools**: Tools that are provided by OpenAI that extend the - model's capabilities, like [web search](/docs/guides/tools-web-search) - or [file search](/docs/guides/tools-file-search). Learn more about - [built-in tools](/docs/guides/tools). + model's capabilities, like [web search](https://platform.openai.com/docs/guides/tools-web-search) + or [file search](https://platform.openai.com/docs/guides/tools-file-search). Learn more about + [built-in tools](https://platform.openai.com/docs/guides/tools). - **Function calls (custom tools)**: Functions that are defined by you, enabling the model to call your own code. Learn more about - [function calling](/docs/guides/function-calling). + [function calling](https://platform.openai.com/docs/guides/function-calling). - - `Function object { name, parameters, strict, 3 more }` + - `Function object { name, parameters, strict, 5 more }` Defines a function in your own code the model can choose to call. Learn more about [function calling](https://platform.openai.com/docs/guides/function-calling). @@ -12419,7 +13095,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `strict: boolean` - Whether to enforce strict parameter validation. Default `true`. + Whether strict parameter validation is enforced for this function tool. - `type: "function"` @@ -12427,6 +13103,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"function"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this function is deferred and loaded via tool search. @@ -12435,6 +13119,10 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A description of the function. Used by the model to determine whether or not to call the function. + - `output_schema: optional map[unknown]` + + A JSON schema object describing the JSON value encoded in string outputs for this function. + - `FileSearch object { type, vector_store_ids, filters, 2 more }` A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](https://platform.openai.com/docs/guides/tools-file-search). @@ -12605,7 +13293,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `WebSearch object { type, filters, search_context_size, user_location }` Search the Internet for sources related to the prompt. Learn more about the - [web search tool](/docs/guides/tools-web-search). + [web search tool](https://platform.openai.com/docs/guides/tools-web-search). - `type: "web_search" or "web_search_2025_08_26"` @@ -12662,10 +13350,10 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"approximate"` - - `Mcp object { server_label, type, allowed_tools, 8 more }` + - `Mcp object { server_label, type, allowed_callers, 9 more }` Give the model access to additional tools via remote Model Context Protocol - (MCP) servers. [Learn more about MCP](/docs/guides/tools-remote-mcp). + (MCP) servers. [Learn more about MCP](https://platform.openai.com/docs/guides/tools-remote-mcp). - `server_label: string` @@ -12677,6 +13365,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"mcp"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `allowed_tools: optional array of string or object { read_only, tool_names }` List of allowed tool names or a filter object. @@ -12709,7 +13405,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Identifier for service connectors, like those available in ChatGPT. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more - about service connectors [here](/docs/guides/tools-remote-mcp#connectors). + about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors). Currently supported `connector_id` values are: @@ -12809,7 +13505,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The Secure MCP Tunnel ID to use instead of a direct server URL. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. - - `CodeInterpreter object { container, type }` + - `CodeInterpreter object { container, type, allowed_callers }` A tool that runs Python code to help generate a response to a prompt. @@ -12895,6 +13591,22 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"code_interpreter"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + + - `ProgrammaticToolCalling object { type }` + + - `type: "programmatic_tool_calling"` + + The type of the tool. Always `programmatic_tool_calling`. + + - `"programmatic_tool_calling"` + - `ImageGeneration object { type, action, background, 9 more }` A tool that generates images using the GPT image models. @@ -12917,8 +13629,19 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `background: optional "transparent" or "opaque" or "auto"` - Background type for the generated image. One of `transparent`, - `opaque`, or `auto`. Default: `auto`. + Allows to set transparency for the background of the generated image(s). + This parameter is only supported for GPT image models that support + transparent backgrounds. Must be one of `transparent`, `opaque`, or + `auto` (default value). When `auto` is used, the model will + automatically determine the best background for the image. + + `gpt-image-2` and `gpt-image-2-2026-04-21` do not support + transparent backgrounds. Requests with `background` set to + `transparent` will return an error for these models; use `opaque` or + `auto` instead. + + If `transparent`, the output format needs to support transparency, + so it should be set to either `png` (default value) or `webp`. - `"transparent"` @@ -12947,13 +13670,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Base64-encoded mask image. - - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5"` + - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-2" or 3 more` The image generation model to use. Default: `gpt-image-1`. - `string` - - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5"` + - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-2" or 3 more` The image generation model to use. Default: `gpt-image-1`. @@ -12961,8 +13684,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"gpt-image-1-mini"` + - `"gpt-image-2"` + + - `"gpt-image-2-2026-04-21"` + - `"gpt-image-1.5"` + - `"chatgpt-image-latest"` + - `moderation: optional "auto" or "low"` Moderation level for the generated image. Default: `auto`. @@ -13031,7 +13760,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"local_shell"` - - `Shell object { type, environment }` + - `Shell object { type, allowed_callers, environment }` A tool that allows the model to execute shell commands. @@ -13041,6 +13770,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"shell"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `environment: optional ContainerAuto or LocalEnvironment or ContainerReference` - `ContainerAuto object { type, file_ids, memory_limit, 2 more }` @@ -13167,9 +13904,9 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"container_reference"` - - `Custom object { name, type, defer_loading, 2 more }` + - `Custom object { name, type, allowed_callers, 3 more }` - A custom tool that processes input using a specified format. Learn more about [custom tools](/docs/guides/function-calling#custom-tools) + A custom tool that processes input using a specified format. Learn more about [custom tools](https://platform.openai.com/docs/guides/function-calling#custom-tools) - `name: string` @@ -13181,6 +13918,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"custom"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this tool should be deferred and discovered via tool search. @@ -13237,11 +13982,11 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The namespace name used in tool calls (for example, `crm`). - - `tools: array of object { name, type, defer_loading, 3 more } or object { name, type, defer_loading, 2 more }` + - `tools: array of object { name, type, allowed_callers, 5 more } or object { name, type, allowed_callers, 3 more }` The function/custom tools available inside this namespace. - - `Function object { name, type, defer_loading, 3 more }` + - `Function object { name, type, allowed_callers, 5 more }` - `name: string` @@ -13249,19 +13994,33 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"function"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this function should be deferred and discovered via tool search. - `description: optional string` + - `output_schema: optional map[unknown]` + + A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs. + - `parameters: optional unknown` - `strict: optional boolean` - - `Custom object { name, type, defer_loading, 2 more }` + Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise. + + - `Custom object { name, type, allowed_callers, 3 more }` - A custom tool that processes input using a specified format. Learn more about [custom tools](/docs/guides/function-calling#custom-tools) + A custom tool that processes input using a specified format. Learn more about [custom tools](https://platform.openai.com/docs/guides/function-calling#custom-tools) - `name: string` @@ -13273,6 +14032,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"custom"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this tool should be deferred and discovered via tool search. @@ -13371,7 +14138,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`. - - `ApplyPatch object { type }` + - `ApplyPatch object { type, allowed_callers }` Allows the assistant to create, delete, or update files using unified diffs. @@ -13381,6 +14148,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"apply_patch"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `top_p: optional number` An alternative to temperature for nucleus sampling; 1.0 includes all tokens. @@ -13522,7 +14297,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Determines what populates the `item` namespace in the data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -13538,7 +14313,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -13564,7 +14339,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Determines what populates the `item` namespace in this run's data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -13580,7 +14355,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -13592,7 +14367,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_id"` - - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }` + - `StoredCompletions object { type, created_after, created_before, 3 more }` A StoredCompletionsRunDataSource configuration describing a set of filters @@ -13637,7 +14412,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - - `TemplateInputMessages object { template, type }` + - `Template object { template, type }` - `template: array of EasyInputMessage or object { content, role, type }` @@ -13665,7 +14440,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A list of one or many input items to the model, containing different content types. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -13679,9 +14454,19 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"input_text"` - - `ResponseInputImage object { detail, type, file_id, image_url }` + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` - An image input to the model. Learn about [image inputs](/docs/guides/vision). + - `ResponseInputImage object { detail, type, file_id, 2 more }` + + An image input to the model. Learn about [image inputs](https://platform.openai.com/docs/guides/vision). - `detail: "low" or "high" or "auto" or "original"` @@ -13709,7 +14494,17 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL. - - `ResponseInputFile object { type, detail, file_data, 3 more }` + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + + - `ResponseInputFile object { type, detail, file_data, 4 more }` A file input to the model. @@ -13719,9 +14514,11 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"input_file"` - - `detail: optional "low" or "high"` + - `detail: optional "auto" or "low" or "high"` - The detail level of the file to be sent to the model. Use `low` for the default rendering behavior, or `high` to render the file at higher quality. Defaults to `low`. + The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`. + + - `"auto"` - `"low"` @@ -13743,6 +14540,16 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The name of the file to be sent to the model. + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or @@ -13772,7 +14579,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"message"` - - `EvalMessageObject object { content, role, type }` + - `EvalItem object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take @@ -13788,7 +14595,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -13858,7 +14665,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -13923,7 +14730,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"template"` - - `ItemReferenceInputMessages object { item_reference, type }` + - `ItemReference object { item_reference, type }` - `item_reference: string` @@ -13947,16 +14754,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `"none"` @@ -13970,6 +14774,8 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"xhigh"` + - `"max"` + - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject` An object specifying the format that the model must output. @@ -13977,7 +14783,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Setting to `{ "type": "json_schema", "json_schema": {...} }` enables Structured Outputs which ensures the model will match your supplied JSON schema. Learn more in the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` @@ -13996,7 +14802,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `ResponseFormatJSONSchema object { json_schema, type }` JSON Schema response format. Used to generate structured JSON responses. - Learn more about [Structured Outputs](/docs/guides/structured-outputs). + Learn more about [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs). - `json_schema: object { name, description, schema, strict }` @@ -14023,7 +14829,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). - `type: "json_schema"` @@ -14068,13 +14874,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `parameters: optional FunctionParameters` - The parameters the functions accepts, described as a JSON Schema object. See the [guide](/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. + The parameters the functions accepts, described as a JSON Schema object. See the [guide](https://platform.openai.com/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. Omitting `parameters` defines a function with an empty parameter list. - `strict: optional boolean` - Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/docs/guides/function-calling). + Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](https://platform.openai.com/docs/guides/function-calling). - `type: "function"` @@ -14086,7 +14892,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `ResponsesRunDataSource object { source, type, input_messages, 2 more }` + - `Responses object { source, type, input_messages, 2 more }` A ResponsesRunDataSource object describing a model sampling configuration. @@ -14094,7 +14900,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Determines what populates the `item` namespace in this run's data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -14110,7 +14916,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -14122,7 +14928,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_id"` - - `EvalResponsesSource object { type, created_after, created_before, 8 more }` + - `Responses object { type, created_after, created_before, 8 more }` A EvalResponsesSource object describing a run data source configuration. @@ -14154,16 +14960,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `temperature: optional number` @@ -14191,7 +14994,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - - `InputMessagesTemplate object { template, type }` + - `Template object { template, type }` - `template: array of object { content, role } or object { content, role, type }` @@ -14207,7 +15010,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The role of the message (e.g. "system", "assistant", "user"). - - `EvalMessageObject object { content, role, type }` + - `EvalItem object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take @@ -14223,7 +15026,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -14293,7 +15096,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"template"` - - `InputMessagesItemReference object { item_reference, type }` + - `ItemReference object { item_reference, type }` - `item_reference: string` @@ -14317,16 +15120,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `seed: optional number` @@ -14341,8 +15141,8 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Configuration options for a text response from the model. Can be plain text or structured JSON data. Learn more: - - [Text inputs and outputs](/docs/guides/text) - - [Structured Outputs](/docs/guides/structured-outputs) + - [Text inputs and outputs](https://platform.openai.com/docs/guides/text) + - [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs) - `format: optional ResponseFormatTextConfig` @@ -14350,7 +15150,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Configuring `{ "type": "json_schema" }` enables Structured Outputs, which ensures the model will match your supplied JSON schema. Learn more in the - [Structured Outputs guide](/docs/guides/structured-outputs). + [Structured Outputs guide](https://platform.openai.com/docs/guides/structured-outputs). The default format is `{ "type": "text" }` with no additional options. @@ -14367,7 +15167,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `ResponseFormatTextJSONSchemaConfig object { name, schema, type, 2 more }` JSON Schema response format. Used to generate structured JSON responses. - Learn more about [Structured Outputs](/docs/guides/structured-outputs). + Learn more about [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs). - `name: string` @@ -14396,7 +15196,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). - `ResponseFormatJSONObject object { type }` @@ -14405,7 +15205,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ model will not generate JSON without a system or user message instructing it to do so. - - `tools: optional array of object { name, parameters, strict, 3 more } or object { type, vector_store_ids, filters, 2 more } or object { type } or 12 more` + - `tools: optional array of object { name, parameters, strict, 5 more } or object { type, vector_store_ids, filters, 2 more } or object { type } or 13 more` An array of tools the model may call while generating a response. You can specify which tool to use by setting the `tool_choice` parameter. @@ -14413,14 +15213,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The two categories of tools you can provide the model are: - **Built-in tools**: Tools that are provided by OpenAI that extend the - model's capabilities, like [web search](/docs/guides/tools-web-search) - or [file search](/docs/guides/tools-file-search). Learn more about - [built-in tools](/docs/guides/tools). + model's capabilities, like [web search](https://platform.openai.com/docs/guides/tools-web-search) + or [file search](https://platform.openai.com/docs/guides/tools-file-search). Learn more about + [built-in tools](https://platform.openai.com/docs/guides/tools). - **Function calls (custom tools)**: Functions that are defined by you, enabling the model to call your own code. Learn more about - [function calling](/docs/guides/function-calling). + [function calling](https://platform.openai.com/docs/guides/function-calling). - - `Function object { name, parameters, strict, 3 more }` + - `Function object { name, parameters, strict, 5 more }` Defines a function in your own code the model can choose to call. Learn more about [function calling](https://platform.openai.com/docs/guides/function-calling). @@ -14434,7 +15234,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `strict: boolean` - Whether to enforce strict parameter validation. Default `true`. + Whether strict parameter validation is enforced for this function tool. - `type: "function"` @@ -14442,6 +15242,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"function"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this function is deferred and loaded via tool search. @@ -14450,6 +15258,10 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A description of the function. Used by the model to determine whether or not to call the function. + - `output_schema: optional map[unknown]` + + A JSON schema object describing the JSON value encoded in string outputs for this function. + - `FileSearch object { type, vector_store_ids, filters, 2 more }` A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](https://platform.openai.com/docs/guides/tools-file-search). @@ -14620,7 +15432,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `WebSearch object { type, filters, search_context_size, user_location }` Search the Internet for sources related to the prompt. Learn more about the - [web search tool](/docs/guides/tools-web-search). + [web search tool](https://platform.openai.com/docs/guides/tools-web-search). - `type: "web_search" or "web_search_2025_08_26"` @@ -14677,10 +15489,10 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"approximate"` - - `Mcp object { server_label, type, allowed_tools, 8 more }` + - `Mcp object { server_label, type, allowed_callers, 9 more }` Give the model access to additional tools via remote Model Context Protocol - (MCP) servers. [Learn more about MCP](/docs/guides/tools-remote-mcp). + (MCP) servers. [Learn more about MCP](https://platform.openai.com/docs/guides/tools-remote-mcp). - `server_label: string` @@ -14692,6 +15504,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"mcp"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `allowed_tools: optional array of string or object { read_only, tool_names }` List of allowed tool names or a filter object. @@ -14724,7 +15544,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Identifier for service connectors, like those available in ChatGPT. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more - about service connectors [here](/docs/guides/tools-remote-mcp#connectors). + about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors). Currently supported `connector_id` values are: @@ -14824,7 +15644,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The Secure MCP Tunnel ID to use instead of a direct server URL. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. - - `CodeInterpreter object { container, type }` + - `CodeInterpreter object { container, type, allowed_callers }` A tool that runs Python code to help generate a response to a prompt. @@ -14910,6 +15730,22 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"code_interpreter"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + + - `ProgrammaticToolCalling object { type }` + + - `type: "programmatic_tool_calling"` + + The type of the tool. Always `programmatic_tool_calling`. + + - `"programmatic_tool_calling"` + - `ImageGeneration object { type, action, background, 9 more }` A tool that generates images using the GPT image models. @@ -14932,8 +15768,19 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `background: optional "transparent" or "opaque" or "auto"` - Background type for the generated image. One of `transparent`, - `opaque`, or `auto`. Default: `auto`. + Allows to set transparency for the background of the generated image(s). + This parameter is only supported for GPT image models that support + transparent backgrounds. Must be one of `transparent`, `opaque`, or + `auto` (default value). When `auto` is used, the model will + automatically determine the best background for the image. + + `gpt-image-2` and `gpt-image-2-2026-04-21` do not support + transparent backgrounds. Requests with `background` set to + `transparent` will return an error for these models; use `opaque` or + `auto` instead. + + If `transparent`, the output format needs to support transparency, + so it should be set to either `png` (default value) or `webp`. - `"transparent"` @@ -14962,13 +15809,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Base64-encoded mask image. - - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5"` + - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-2" or 3 more` The image generation model to use. Default: `gpt-image-1`. - `string` - - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5"` + - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-2" or 3 more` The image generation model to use. Default: `gpt-image-1`. @@ -14976,8 +15823,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"gpt-image-1-mini"` + - `"gpt-image-2"` + + - `"gpt-image-2-2026-04-21"` + - `"gpt-image-1.5"` + - `"chatgpt-image-latest"` + - `moderation: optional "auto" or "low"` Moderation level for the generated image. Default: `auto`. @@ -15046,7 +15899,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"local_shell"` - - `Shell object { type, environment }` + - `Shell object { type, allowed_callers, environment }` A tool that allows the model to execute shell commands. @@ -15056,6 +15909,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"shell"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `environment: optional ContainerAuto or LocalEnvironment or ContainerReference` - `ContainerAuto object { type, file_ids, memory_limit, 2 more }` @@ -15182,9 +16043,9 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"container_reference"` - - `Custom object { name, type, defer_loading, 2 more }` + - `Custom object { name, type, allowed_callers, 3 more }` - A custom tool that processes input using a specified format. Learn more about [custom tools](/docs/guides/function-calling#custom-tools) + A custom tool that processes input using a specified format. Learn more about [custom tools](https://platform.openai.com/docs/guides/function-calling#custom-tools) - `name: string` @@ -15196,6 +16057,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"custom"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this tool should be deferred and discovered via tool search. @@ -15252,11 +16121,11 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The namespace name used in tool calls (for example, `crm`). - - `tools: array of object { name, type, defer_loading, 3 more } or object { name, type, defer_loading, 2 more }` + - `tools: array of object { name, type, allowed_callers, 5 more } or object { name, type, allowed_callers, 3 more }` The function/custom tools available inside this namespace. - - `Function object { name, type, defer_loading, 3 more }` + - `Function object { name, type, allowed_callers, 5 more }` - `name: string` @@ -15264,19 +16133,33 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"function"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this function should be deferred and discovered via tool search. - `description: optional string` + - `output_schema: optional map[unknown]` + + A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs. + - `parameters: optional unknown` - `strict: optional boolean` - - `Custom object { name, type, defer_loading, 2 more }` + Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise. + + - `Custom object { name, type, allowed_callers, 3 more }` - A custom tool that processes input using a specified format. Learn more about [custom tools](/docs/guides/function-calling#custom-tools) + A custom tool that processes input using a specified format. Learn more about [custom tools](https://platform.openai.com/docs/guides/function-calling#custom-tools) - `name: string` @@ -15288,6 +16171,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"custom"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this tool should be deferred and discovered via tool search. @@ -15386,7 +16277,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`. - - `ApplyPatch object { type }` + - `ApplyPatch object { type, allowed_callers }` Allows the assistant to create, delete, or update files using unified diffs. @@ -15396,6 +16287,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"apply_patch"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `top_p: optional number` An alternative to temperature for nucleus sampling; 1.0 includes all tokens. @@ -15537,7 +16436,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Determines what populates the `item` namespace in the data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -15553,7 +16452,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -15579,7 +16478,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Determines what populates the `item` namespace in this run's data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -15595,7 +16494,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -15607,7 +16506,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_id"` - - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }` + - `StoredCompletions object { type, created_after, created_before, 3 more }` A StoredCompletionsRunDataSource configuration describing a set of filters @@ -15652,7 +16551,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - - `TemplateInputMessages object { template, type }` + - `Template object { template, type }` - `template: array of EasyInputMessage or object { content, role, type }` @@ -15680,7 +16579,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A list of one or many input items to the model, containing different content types. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -15694,9 +16593,19 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"input_text"` - - `ResponseInputImage object { detail, type, file_id, image_url }` + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. - An image input to the model. Learn about [image inputs](/docs/guides/vision). + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + + - `ResponseInputImage object { detail, type, file_id, 2 more }` + + An image input to the model. Learn about [image inputs](https://platform.openai.com/docs/guides/vision). - `detail: "low" or "high" or "auto" or "original"` @@ -15724,7 +16633,17 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL. - - `ResponseInputFile object { type, detail, file_data, 3 more }` + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + + - `ResponseInputFile object { type, detail, file_data, 4 more }` A file input to the model. @@ -15734,9 +16653,11 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"input_file"` - - `detail: optional "low" or "high"` + - `detail: optional "auto" or "low" or "high"` - The detail level of the file to be sent to the model. Use `low` for the default rendering behavior, or `high` to render the file at higher quality. Defaults to `low`. + The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`. + + - `"auto"` - `"low"` @@ -15758,6 +16679,16 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The name of the file to be sent to the model. + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or @@ -15787,7 +16718,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"message"` - - `EvalMessageObject object { content, role, type }` + - `EvalItem object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take @@ -15803,7 +16734,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -15873,7 +16804,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -15938,7 +16869,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"template"` - - `ItemReferenceInputMessages object { item_reference, type }` + - `ItemReference object { item_reference, type }` - `item_reference: string` @@ -15962,16 +16893,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `"none"` @@ -15985,6 +16913,8 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"xhigh"` + - `"max"` + - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject` An object specifying the format that the model must output. @@ -15992,7 +16922,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Setting to `{ "type": "json_schema", "json_schema": {...} }` enables Structured Outputs which ensures the model will match your supplied JSON schema. Learn more in the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` @@ -16011,7 +16941,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `ResponseFormatJSONSchema object { json_schema, type }` JSON Schema response format. Used to generate structured JSON responses. - Learn more about [Structured Outputs](/docs/guides/structured-outputs). + Learn more about [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs). - `json_schema: object { name, description, schema, strict }` @@ -16038,7 +16968,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). - `type: "json_schema"` @@ -16083,13 +17013,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `parameters: optional FunctionParameters` - The parameters the functions accepts, described as a JSON Schema object. See the [guide](/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. + The parameters the functions accepts, described as a JSON Schema object. See the [guide](https://platform.openai.com/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. Omitting `parameters` defines a function with an empty parameter list. - `strict: optional boolean` - Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/docs/guides/function-calling). + Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](https://platform.openai.com/docs/guides/function-calling). - `type: "function"` @@ -16101,7 +17031,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `ResponsesRunDataSource object { source, type, input_messages, 2 more }` + - `Responses object { source, type, input_messages, 2 more }` A ResponsesRunDataSource object describing a model sampling configuration. @@ -16109,7 +17039,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Determines what populates the `item` namespace in this run's data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -16125,7 +17055,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -16137,7 +17067,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_id"` - - `EvalResponsesSource object { type, created_after, created_before, 8 more }` + - `Responses object { type, created_after, created_before, 8 more }` A EvalResponsesSource object describing a run data source configuration. @@ -16169,16 +17099,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `temperature: optional number` @@ -16206,7 +17133,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - - `InputMessagesTemplate object { template, type }` + - `Template object { template, type }` - `template: array of object { content, role } or object { content, role, type }` @@ -16222,7 +17149,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The role of the message (e.g. "system", "assistant", "user"). - - `EvalMessageObject object { content, role, type }` + - `EvalItem object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take @@ -16238,7 +17165,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -16308,7 +17235,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"template"` - - `InputMessagesItemReference object { item_reference, type }` + - `ItemReference object { item_reference, type }` - `item_reference: string` @@ -16332,16 +17259,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `seed: optional number` @@ -16356,8 +17280,8 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Configuration options for a text response from the model. Can be plain text or structured JSON data. Learn more: - - [Text inputs and outputs](/docs/guides/text) - - [Structured Outputs](/docs/guides/structured-outputs) + - [Text inputs and outputs](https://platform.openai.com/docs/guides/text) + - [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs) - `format: optional ResponseFormatTextConfig` @@ -16365,7 +17289,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Configuring `{ "type": "json_schema" }` enables Structured Outputs, which ensures the model will match your supplied JSON schema. Learn more in the - [Structured Outputs guide](/docs/guides/structured-outputs). + [Structured Outputs guide](https://platform.openai.com/docs/guides/structured-outputs). The default format is `{ "type": "text" }` with no additional options. @@ -16382,7 +17306,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `ResponseFormatTextJSONSchemaConfig object { name, schema, type, 2 more }` JSON Schema response format. Used to generate structured JSON responses. - Learn more about [Structured Outputs](/docs/guides/structured-outputs). + Learn more about [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs). - `name: string` @@ -16411,7 +17335,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). - `ResponseFormatJSONObject object { type }` @@ -16420,7 +17344,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ model will not generate JSON without a system or user message instructing it to do so. - - `tools: optional array of object { name, parameters, strict, 3 more } or object { type, vector_store_ids, filters, 2 more } or object { type } or 12 more` + - `tools: optional array of object { name, parameters, strict, 5 more } or object { type, vector_store_ids, filters, 2 more } or object { type } or 13 more` An array of tools the model may call while generating a response. You can specify which tool to use by setting the `tool_choice` parameter. @@ -16428,14 +17352,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The two categories of tools you can provide the model are: - **Built-in tools**: Tools that are provided by OpenAI that extend the - model's capabilities, like [web search](/docs/guides/tools-web-search) - or [file search](/docs/guides/tools-file-search). Learn more about - [built-in tools](/docs/guides/tools). + model's capabilities, like [web search](https://platform.openai.com/docs/guides/tools-web-search) + or [file search](https://platform.openai.com/docs/guides/tools-file-search). Learn more about + [built-in tools](https://platform.openai.com/docs/guides/tools). - **Function calls (custom tools)**: Functions that are defined by you, enabling the model to call your own code. Learn more about - [function calling](/docs/guides/function-calling). + [function calling](https://platform.openai.com/docs/guides/function-calling). - - `Function object { name, parameters, strict, 3 more }` + - `Function object { name, parameters, strict, 5 more }` Defines a function in your own code the model can choose to call. Learn more about [function calling](https://platform.openai.com/docs/guides/function-calling). @@ -16449,7 +17373,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `strict: boolean` - Whether to enforce strict parameter validation. Default `true`. + Whether strict parameter validation is enforced for this function tool. - `type: "function"` @@ -16457,6 +17381,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"function"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this function is deferred and loaded via tool search. @@ -16465,6 +17397,10 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A description of the function. Used by the model to determine whether or not to call the function. + - `output_schema: optional map[unknown]` + + A JSON schema object describing the JSON value encoded in string outputs for this function. + - `FileSearch object { type, vector_store_ids, filters, 2 more }` A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](https://platform.openai.com/docs/guides/tools-file-search). @@ -16635,7 +17571,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `WebSearch object { type, filters, search_context_size, user_location }` Search the Internet for sources related to the prompt. Learn more about the - [web search tool](/docs/guides/tools-web-search). + [web search tool](https://platform.openai.com/docs/guides/tools-web-search). - `type: "web_search" or "web_search_2025_08_26"` @@ -16692,10 +17628,10 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"approximate"` - - `Mcp object { server_label, type, allowed_tools, 8 more }` + - `Mcp object { server_label, type, allowed_callers, 9 more }` Give the model access to additional tools via remote Model Context Protocol - (MCP) servers. [Learn more about MCP](/docs/guides/tools-remote-mcp). + (MCP) servers. [Learn more about MCP](https://platform.openai.com/docs/guides/tools-remote-mcp). - `server_label: string` @@ -16707,6 +17643,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"mcp"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `allowed_tools: optional array of string or object { read_only, tool_names }` List of allowed tool names or a filter object. @@ -16739,7 +17683,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Identifier for service connectors, like those available in ChatGPT. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more - about service connectors [here](/docs/guides/tools-remote-mcp#connectors). + about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors). Currently supported `connector_id` values are: @@ -16839,7 +17783,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The Secure MCP Tunnel ID to use instead of a direct server URL. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. - - `CodeInterpreter object { container, type }` + - `CodeInterpreter object { container, type, allowed_callers }` A tool that runs Python code to help generate a response to a prompt. @@ -16925,6 +17869,22 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"code_interpreter"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + + - `ProgrammaticToolCalling object { type }` + + - `type: "programmatic_tool_calling"` + + The type of the tool. Always `programmatic_tool_calling`. + + - `"programmatic_tool_calling"` + - `ImageGeneration object { type, action, background, 9 more }` A tool that generates images using the GPT image models. @@ -16947,8 +17907,19 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `background: optional "transparent" or "opaque" or "auto"` - Background type for the generated image. One of `transparent`, - `opaque`, or `auto`. Default: `auto`. + Allows to set transparency for the background of the generated image(s). + This parameter is only supported for GPT image models that support + transparent backgrounds. Must be one of `transparent`, `opaque`, or + `auto` (default value). When `auto` is used, the model will + automatically determine the best background for the image. + + `gpt-image-2` and `gpt-image-2-2026-04-21` do not support + transparent backgrounds. Requests with `background` set to + `transparent` will return an error for these models; use `opaque` or + `auto` instead. + + If `transparent`, the output format needs to support transparency, + so it should be set to either `png` (default value) or `webp`. - `"transparent"` @@ -16977,13 +17948,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Base64-encoded mask image. - - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5"` + - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-2" or 3 more` The image generation model to use. Default: `gpt-image-1`. - `string` - - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5"` + - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-2" or 3 more` The image generation model to use. Default: `gpt-image-1`. @@ -16991,8 +17962,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"gpt-image-1-mini"` + - `"gpt-image-2"` + + - `"gpt-image-2-2026-04-21"` + - `"gpt-image-1.5"` + - `"chatgpt-image-latest"` + - `moderation: optional "auto" or "low"` Moderation level for the generated image. Default: `auto`. @@ -17061,7 +18038,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"local_shell"` - - `Shell object { type, environment }` + - `Shell object { type, allowed_callers, environment }` A tool that allows the model to execute shell commands. @@ -17071,6 +18048,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"shell"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `environment: optional ContainerAuto or LocalEnvironment or ContainerReference` - `ContainerAuto object { type, file_ids, memory_limit, 2 more }` @@ -17197,9 +18182,9 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"container_reference"` - - `Custom object { name, type, defer_loading, 2 more }` + - `Custom object { name, type, allowed_callers, 3 more }` - A custom tool that processes input using a specified format. Learn more about [custom tools](/docs/guides/function-calling#custom-tools) + A custom tool that processes input using a specified format. Learn more about [custom tools](https://platform.openai.com/docs/guides/function-calling#custom-tools) - `name: string` @@ -17211,6 +18196,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"custom"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this tool should be deferred and discovered via tool search. @@ -17267,11 +18260,11 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The namespace name used in tool calls (for example, `crm`). - - `tools: array of object { name, type, defer_loading, 3 more } or object { name, type, defer_loading, 2 more }` + - `tools: array of object { name, type, allowed_callers, 5 more } or object { name, type, allowed_callers, 3 more }` The function/custom tools available inside this namespace. - - `Function object { name, type, defer_loading, 3 more }` + - `Function object { name, type, allowed_callers, 5 more }` - `name: string` @@ -17279,19 +18272,33 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"function"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this function should be deferred and discovered via tool search. - `description: optional string` + - `output_schema: optional map[unknown]` + + A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs. + - `parameters: optional unknown` - `strict: optional boolean` - - `Custom object { name, type, defer_loading, 2 more }` + Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise. + + - `Custom object { name, type, allowed_callers, 3 more }` - A custom tool that processes input using a specified format. Learn more about [custom tools](/docs/guides/function-calling#custom-tools) + A custom tool that processes input using a specified format. Learn more about [custom tools](https://platform.openai.com/docs/guides/function-calling#custom-tools) - `name: string` @@ -17303,6 +18310,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"custom"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this tool should be deferred and discovered via tool search. @@ -17401,7 +18416,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`. - - `ApplyPatch object { type }` + - `ApplyPatch object { type, allowed_callers }` Allows the assistant to create, delete, or update files using unified diffs. @@ -17411,6 +18426,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"apply_patch"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `top_p: optional number` An alternative to temperature for nucleus sampling; 1.0 includes all tokens. @@ -17552,7 +18575,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Determines what populates the `item` namespace in the data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -17568,7 +18591,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -17594,7 +18617,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Determines what populates the `item` namespace in this run's data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -17610,7 +18633,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -17622,7 +18645,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_id"` - - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }` + - `StoredCompletions object { type, created_after, created_before, 3 more }` A StoredCompletionsRunDataSource configuration describing a set of filters @@ -17667,7 +18690,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - - `TemplateInputMessages object { template, type }` + - `Template object { template, type }` - `template: array of EasyInputMessage or object { content, role, type }` @@ -17695,7 +18718,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A list of one or many input items to the model, containing different content types. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -17709,9 +18732,19 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"input_text"` - - `ResponseInputImage object { detail, type, file_id, image_url }` + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` - An image input to the model. Learn about [image inputs](/docs/guides/vision). + - `ResponseInputImage object { detail, type, file_id, 2 more }` + + An image input to the model. Learn about [image inputs](https://platform.openai.com/docs/guides/vision). - `detail: "low" or "high" or "auto" or "original"` @@ -17739,7 +18772,17 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL. - - `ResponseInputFile object { type, detail, file_data, 3 more }` + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + + - `ResponseInputFile object { type, detail, file_data, 4 more }` A file input to the model. @@ -17749,9 +18792,11 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"input_file"` - - `detail: optional "low" or "high"` + - `detail: optional "auto" or "low" or "high"` - The detail level of the file to be sent to the model. Use `low` for the default rendering behavior, or `high` to render the file at higher quality. Defaults to `low`. + The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`. + + - `"auto"` - `"low"` @@ -17773,6 +18818,16 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The name of the file to be sent to the model. + - `prompt_cache_breakpoint: optional object { mode }` + + Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. + + - `mode: "explicit"` + + The breakpoint mode. Always `explicit`. + + - `"explicit"` + - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or @@ -17802,7 +18857,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"message"` - - `EvalMessageObject object { content, role, type }` + - `EvalItem object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take @@ -17818,7 +18873,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -17888,7 +18943,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -17953,7 +19008,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"template"` - - `ItemReferenceInputMessages object { item_reference, type }` + - `ItemReference object { item_reference, type }` - `item_reference: string` @@ -17977,16 +19032,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `"none"` @@ -18000,6 +19052,8 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"xhigh"` + - `"max"` + - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject` An object specifying the format that the model must output. @@ -18007,7 +19061,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Setting to `{ "type": "json_schema", "json_schema": {...} }` enables Structured Outputs which ensures the model will match your supplied JSON schema. Learn more in the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` @@ -18026,7 +19080,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `ResponseFormatJSONSchema object { json_schema, type }` JSON Schema response format. Used to generate structured JSON responses. - Learn more about [Structured Outputs](/docs/guides/structured-outputs). + Learn more about [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs). - `json_schema: object { name, description, schema, strict }` @@ -18053,7 +19107,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). - `type: "json_schema"` @@ -18098,13 +19152,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `parameters: optional FunctionParameters` - The parameters the functions accepts, described as a JSON Schema object. See the [guide](/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. + The parameters the functions accepts, described as a JSON Schema object. See the [guide](https://platform.openai.com/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. Omitting `parameters` defines a function with an empty parameter list. - `strict: optional boolean` - Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/docs/guides/function-calling). + Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](https://platform.openai.com/docs/guides/function-calling). - `type: "function"` @@ -18116,7 +19170,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `ResponsesRunDataSource object { source, type, input_messages, 2 more }` + - `Responses object { source, type, input_messages, 2 more }` A ResponsesRunDataSource object describing a model sampling configuration. @@ -18124,7 +19178,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Determines what populates the `item` namespace in this run's data source. - - `EvalJSONLFileContentSource object { content, type }` + - `FileContent object { content, type }` - `content: array of object { item, sample }` @@ -18140,7 +19194,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_content"` - - `EvalJSONLFileIDSource object { id, type }` + - `FileID object { id, type }` - `id: string` @@ -18152,7 +19206,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"file_id"` - - `EvalResponsesSource object { type, created_after, created_before, 8 more }` + - `Responses object { type, created_after, created_before, 8 more }` A EvalResponsesSource object describing a run data source configuration. @@ -18184,16 +19238,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `temperature: optional number` @@ -18221,7 +19272,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - - `InputMessagesTemplate object { template, type }` + - `Template object { template, type }` - `template: array of object { content, role } or object { content, role, type }` @@ -18237,7 +19288,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The role of the message (e.g. "system", "assistant", "user"). - - `EvalMessageObject object { content, role, type }` + - `EvalItem object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take @@ -18253,7 +19304,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A text input to the model. - - `ResponseInputText object { text, type }` + - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. @@ -18323,7 +19374,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"template"` - - `InputMessagesItemReference object { item_reference, type }` + - `ItemReference object { item_reference, type }` - `item_reference: string` @@ -18347,16 +19398,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `reasoning_effort: optional ReasoningEffort` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Constrains effort on reasoning for reasoning models. Currently supported + values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. + Reducing reasoning effort can result in faster responses and fewer tokens + used on reasoning in a response. Not all reasoning models support every + value. See the + [reasoning guide](https://platform.openai.com/docs/guides/reasoning) + for model-specific support. - `seed: optional number` @@ -18371,8 +19419,8 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Configuration options for a text response from the model. Can be plain text or structured JSON data. Learn more: - - [Text inputs and outputs](/docs/guides/text) - - [Structured Outputs](/docs/guides/structured-outputs) + - [Text inputs and outputs](https://platform.openai.com/docs/guides/text) + - [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs) - `format: optional ResponseFormatTextConfig` @@ -18380,7 +19428,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Configuring `{ "type": "json_schema" }` enables Structured Outputs, which ensures the model will match your supplied JSON schema. Learn more in the - [Structured Outputs guide](/docs/guides/structured-outputs). + [Structured Outputs guide](https://platform.openai.com/docs/guides/structured-outputs). The default format is `{ "type": "text" }` with no additional options. @@ -18397,7 +19445,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `ResponseFormatTextJSONSchemaConfig object { name, schema, type, 2 more }` JSON Schema response format. Used to generate structured JSON responses. - Learn more about [Structured Outputs](/docs/guides/structured-outputs). + Learn more about [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs). - `name: string` @@ -18426,7 +19474,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs - guide](/docs/guides/structured-outputs). + guide](https://platform.openai.com/docs/guides/structured-outputs). - `ResponseFormatJSONObject object { type }` @@ -18435,7 +19483,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ model will not generate JSON without a system or user message instructing it to do so. - - `tools: optional array of object { name, parameters, strict, 3 more } or object { type, vector_store_ids, filters, 2 more } or object { type } or 12 more` + - `tools: optional array of object { name, parameters, strict, 5 more } or object { type, vector_store_ids, filters, 2 more } or object { type } or 13 more` An array of tools the model may call while generating a response. You can specify which tool to use by setting the `tool_choice` parameter. @@ -18443,14 +19491,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The two categories of tools you can provide the model are: - **Built-in tools**: Tools that are provided by OpenAI that extend the - model's capabilities, like [web search](/docs/guides/tools-web-search) - or [file search](/docs/guides/tools-file-search). Learn more about - [built-in tools](/docs/guides/tools). + model's capabilities, like [web search](https://platform.openai.com/docs/guides/tools-web-search) + or [file search](https://platform.openai.com/docs/guides/tools-file-search). Learn more about + [built-in tools](https://platform.openai.com/docs/guides/tools). - **Function calls (custom tools)**: Functions that are defined by you, enabling the model to call your own code. Learn more about - [function calling](/docs/guides/function-calling). + [function calling](https://platform.openai.com/docs/guides/function-calling). - - `Function object { name, parameters, strict, 3 more }` + - `Function object { name, parameters, strict, 5 more }` Defines a function in your own code the model can choose to call. Learn more about [function calling](https://platform.openai.com/docs/guides/function-calling). @@ -18464,7 +19512,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `strict: boolean` - Whether to enforce strict parameter validation. Default `true`. + Whether strict parameter validation is enforced for this function tool. - `type: "function"` @@ -18472,6 +19520,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"function"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this function is deferred and loaded via tool search. @@ -18480,6 +19536,10 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ A description of the function. Used by the model to determine whether or not to call the function. + - `output_schema: optional map[unknown]` + + A JSON schema object describing the JSON value encoded in string outputs for this function. + - `FileSearch object { type, vector_store_ids, filters, 2 more }` A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](https://platform.openai.com/docs/guides/tools-file-search). @@ -18650,7 +19710,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `WebSearch object { type, filters, search_context_size, user_location }` Search the Internet for sources related to the prompt. Learn more about the - [web search tool](/docs/guides/tools-web-search). + [web search tool](https://platform.openai.com/docs/guides/tools-web-search). - `type: "web_search" or "web_search_2025_08_26"` @@ -18707,10 +19767,10 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"approximate"` - - `Mcp object { server_label, type, allowed_tools, 8 more }` + - `Mcp object { server_label, type, allowed_callers, 9 more }` Give the model access to additional tools via remote Model Context Protocol - (MCP) servers. [Learn more about MCP](/docs/guides/tools-remote-mcp). + (MCP) servers. [Learn more about MCP](https://platform.openai.com/docs/guides/tools-remote-mcp). - `server_label: string` @@ -18722,6 +19782,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"mcp"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `allowed_tools: optional array of string or object { read_only, tool_names }` List of allowed tool names or a filter object. @@ -18754,7 +19822,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Identifier for service connectors, like those available in ChatGPT. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more - about service connectors [here](/docs/guides/tools-remote-mcp#connectors). + about service connectors [here](https://platform.openai.com/docs/guides/tools-remote-mcp#connectors). Currently supported `connector_id` values are: @@ -18854,7 +19922,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The Secure MCP Tunnel ID to use instead of a direct server URL. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. - - `CodeInterpreter object { container, type }` + - `CodeInterpreter object { container, type, allowed_callers }` A tool that runs Python code to help generate a response to a prompt. @@ -18940,6 +20008,22 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"code_interpreter"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + + - `ProgrammaticToolCalling object { type }` + + - `type: "programmatic_tool_calling"` + + The type of the tool. Always `programmatic_tool_calling`. + + - `"programmatic_tool_calling"` + - `ImageGeneration object { type, action, background, 9 more }` A tool that generates images using the GPT image models. @@ -18962,8 +20046,19 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `background: optional "transparent" or "opaque" or "auto"` - Background type for the generated image. One of `transparent`, - `opaque`, or `auto`. Default: `auto`. + Allows to set transparency for the background of the generated image(s). + This parameter is only supported for GPT image models that support + transparent backgrounds. Must be one of `transparent`, `opaque`, or + `auto` (default value). When `auto` is used, the model will + automatically determine the best background for the image. + + `gpt-image-2` and `gpt-image-2-2026-04-21` do not support + transparent backgrounds. Requests with `background` set to + `transparent` will return an error for these models; use `opaque` or + `auto` instead. + + If `transparent`, the output format needs to support transparency, + so it should be set to either `png` (default value) or `webp`. - `"transparent"` @@ -18992,13 +20087,13 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ Base64-encoded mask image. - - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5"` + - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-2" or 3 more` The image generation model to use. Default: `gpt-image-1`. - `string` - - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5"` + - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-2" or 3 more` The image generation model to use. Default: `gpt-image-1`. @@ -19006,8 +20101,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"gpt-image-1-mini"` + - `"gpt-image-2"` + + - `"gpt-image-2-2026-04-21"` + - `"gpt-image-1.5"` + - `"chatgpt-image-latest"` + - `moderation: optional "auto" or "low"` Moderation level for the generated image. Default: `auto`. @@ -19076,7 +20177,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"local_shell"` - - `Shell object { type, environment }` + - `Shell object { type, allowed_callers, environment }` A tool that allows the model to execute shell commands. @@ -19086,6 +20187,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"shell"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `environment: optional ContainerAuto or LocalEnvironment or ContainerReference` - `ContainerAuto object { type, file_ids, memory_limit, 2 more }` @@ -19212,9 +20321,9 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"container_reference"` - - `Custom object { name, type, defer_loading, 2 more }` + - `Custom object { name, type, allowed_callers, 3 more }` - A custom tool that processes input using a specified format. Learn more about [custom tools](/docs/guides/function-calling#custom-tools) + A custom tool that processes input using a specified format. Learn more about [custom tools](https://platform.openai.com/docs/guides/function-calling#custom-tools) - `name: string` @@ -19226,6 +20335,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"custom"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this tool should be deferred and discovered via tool search. @@ -19282,11 +20399,11 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The namespace name used in tool calls (for example, `crm`). - - `tools: array of object { name, type, defer_loading, 3 more } or object { name, type, defer_loading, 2 more }` + - `tools: array of object { name, type, allowed_callers, 5 more } or object { name, type, allowed_callers, 3 more }` The function/custom tools available inside this namespace. - - `Function object { name, type, defer_loading, 3 more }` + - `Function object { name, type, allowed_callers, 5 more }` - `name: string` @@ -19294,19 +20411,33 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"function"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this function should be deferred and discovered via tool search. - `description: optional string` + - `output_schema: optional map[unknown]` + + A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs. + - `parameters: optional unknown` - `strict: optional boolean` - - `Custom object { name, type, defer_loading, 2 more }` + Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise. + + - `Custom object { name, type, allowed_callers, 3 more }` - A custom tool that processes input using a specified format. Learn more about [custom tools](/docs/guides/function-calling#custom-tools) + A custom tool that processes input using a specified format. Learn more about [custom tools](https://platform.openai.com/docs/guides/function-calling#custom-tools) - `name: string` @@ -19318,6 +20449,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"custom"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `defer_loading: optional boolean` Whether this tool should be deferred and discovered via tool search. @@ -19416,7 +20555,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`. - - `ApplyPatch object { type }` + - `ApplyPatch object { type, allowed_callers }` Allows the assistant to create, delete, or update files using unified diffs. @@ -19426,6 +20565,14 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ - `"apply_patch"` + - `allowed_callers: optional array of "direct" or "programmatic"` + + The tool invocation context(s). + + - `"direct"` + + - `"programmatic"` + - `top_p: optional number` An alternative to temperature for nucleus sampling; 1.0 includes all tokens. @@ -19557,7 +20704,7 @@ curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \ **get** `/evals/{eval_id}/runs/{run_id}/output_items` -Get a list of output items for an evaluation run. +Get eval run output items ### Path Parameters @@ -19917,7 +21064,7 @@ curl https://api.openai.com/v1/evals/egroup_67abd54d9b0081909a86353f6fb9317a/run **get** `/evals/{eval_id}/runs/{run_id}/output_items/{output_item_id}` -Get an evaluation run output item by ID. +Get an output item of an eval run ### Path Parameters