Structured model outputs

import { snippetRefusalsChatCompletionsApi, snippetRefusalsResponsesApi, } from "./inline-examples";

export const snippetRefusalApiResponseChatCompletionsApi = { json: { "id": "chatcmpl-9nYAG9LPNonX8DAyrkwYfemr3C8HC", "object": "chat.completion", "created": 1721596428, "model": "gpt-4o-2024-08-06", "choices": [ { "index": 0, "message": { "role": "assistant", // highlight-start "refusal": "I'm sorry, I cannot assist with that request." // highlight-end }, "logprobs": null, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 81, "completion_tokens": 11, "total_tokens": 92, "completion_tokens_details": { "reasoning_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } }, "system_fingerprint": "fp_3407719c7f" }.trim(), }; export const snippetRefusalApiResponseResponsesApi = { json: { "id": "resp_1234567890", "object": "response", "created_at": 1721596428, "status": "completed", "completed_at": 1721596429, "error": null, "incomplete_details": null, "input": [], "instructions": null, "max_output_tokens": null, "model": "gpt-4o-2024-08-06", "output": [{ "id": "msg_1234567890", "type": "message", "role": "assistant", "content": [ // highlight-start { "type": "refusal", "refusal": "I'm sorry, I cannot assist with that request." } // highlight-end ] }], "usage": { "input_tokens": 81, "output_tokens": 11, "total_tokens": 92, "output_tokens_details": { "reasoning_tokens": 0, } }, }.trim(), };

JSON is one of the most widely used formats in the world for applications to exchange data.

Structured Outputs is a feature that ensures the model will always generate responses that adhere to your supplied JSON Schema, so you don't need to worry about the model omitting a required key, or hallucinating an invalid enum value.

Some benefits of Structured Outputs include:

1. Reliable type-safety: No need to validate or retry incorrectly formatted responses
2. Explicit refusals: Safety-based model refusals are now programmatically detectable
3. Simpler prompting: No need for strongly worded prompts to achieve consistent formatting

In addition to supporting JSON Schema in the REST API, the OpenAI SDKs for Python and JavaScript also make it easy to define object schemas using Pydantic and Zod respectively. Below, you can see how to extract information from unstructured text that conforms to a schema defined in code.

Supported models

Structured Outputs is available in our latest large language models, starting with GPT-4o. Older models like gpt-4-turbo and earlier may use JSON mode instead.

When to use Structured Outputs via function calling vs via <span className="monospace">text.format

Structured Outputs is available in two forms in the OpenAI API:

1. When using function calling
2. When using a json_schema response format

Function calling is useful when you are building an application that bridges the models and functionality of your application.

For example, you can give the model access to functions that query a database in order to build an AI assistant that can help users with their orders, or functions that can interact with the UI.

Conversely, Structured Outputs via response_format are more suitable when you want to indicate a structured schema for use when the model responds to the user, rather than when the model calls a tool.

For example, if you are building a math tutoring application, you might want the assistant to respond to your user using a specific JSON Schema so that you can generate a UI that displays different parts of the model's output in distinct ways.

Put simply:

• If you are connecting the model to tools, functions, data, etc. in your system, then you should use function calling - If you want to structure the model's output when it responds to the user, then you should use a structured text.format

The remainder of this guide will focus on non-function calling use cases in the Responses API. To learn more about how to use Structured Outputs with function calling, check out the Function Calling guide.

Structured Outputs vs JSON mode

Structured Outputs is the evolution of JSON mode. While both ensure valid JSON is produced, only Structured Outputs ensure schema adherence. Both Structured Outputs and JSON mode are supported in the Responses API, Chat Completions API, Assistants API, Fine-tuning API and Batch API.

We recommend always using Structured Outputs instead of JSON mode when possible.

However, Structured Outputs with response_format: {type: "json_schema", ...} is only supported with the gpt-4o-mini, gpt-4o-mini-2024-07-18, and gpt-4o-2024-08-06 model snapshots and later.

Examples

How to use Structured Outputs with <span className="monospace">text.format

Refusals with Structured Outputs

When using Structured Outputs with user-generated input, OpenAI models may occasionally refuse to fulfill the request for safety reasons. Since a refusal does not necessarily follow the schema you have supplied in response_format, the API response will include a new field called refusal to indicate that the model refused to fulfill the request.

When the refusal property appears in your output object, you might present the refusal in your UI, or include conditional logic in code that consumes the response to handle the case of a refused request.

The API response from a refusal will look something like this:

Tips and best practices

Handling user-generated input

If your application is using user-generated input, make sure your prompt includes instructions on how to handle situations where the input cannot result in a valid response.

The model will always try to adhere to the provided schema, which can result in hallucinations if the input is completely unrelated to the schema.

You could include language in your prompt to specify that you want to return empty parameters, or a specific sentence, if the model detects that the input is incompatible with the task.

Handling mistakes

Structured Outputs can still contain mistakes. If you see mistakes, try adjusting your instructions, providing examples in the system instructions, or splitting tasks into simpler subtasks. Refer to the prompt engineering guide for more guidance on how to tweak your inputs.

Avoid JSON schema divergence

To prevent your JSON Schema and corresponding types in your programming language from diverging, we strongly recommend using the native Pydantic/zod sdk support.

If you prefer to specify the JSON schema directly, you could add CI rules that flag when either the JSON schema or underlying data objects are edited, or add a CI step that auto-generates the JSON Schema from type definitions (or vice-versa).

Streaming

Supported schemas

JSON mode

JSON mode is a more basic version of the Structured Outputs feature. While JSON mode ensures that model output is valid JSON, Structured Outputs reliably matches the model's output to the schema you specify. We recommend you use Structured Outputs if it is supported for your use case.

When JSON mode is turned on, the model's output is ensured to be valid JSON, except for in some edge cases that you should detect and handle appropriately.

To turn on JSON mode with the Responses API you can set the text.format to { "type": "json_object" }. If you are using function calling, JSON mode is always turned on.

Important notes:

• When using JSON mode, you must always instruct the model to produce JSON via some message in the conversation, for example via your system message. If you don't include an explicit instruction to generate JSON, the model may generate an unending stream of whitespace and the request may run continually until it reaches the token limit. To help ensure you don't forget, the API will throw an error if the string "JSON" does not appear somewhere in the context.
• JSON mode will not guarantee the output matches any specific schema, only that it is valid and parses without errors. You should use Structured Outputs to ensure it matches your schema, or if that is not possible, you should use a validation library and potentially retries to ensure that the output matches your desired schema.
• Your application must detect and handle the edge cases that can result in the model output not being a complete JSON object (see below)

Handling edge cases

Resources

To learn more about Structured Outputs, we recommend browsing the following resources:

• Check out our introductory cookbook on Structured Outputs
• Learn how to build multi-agent systems with Structured Outputs