diff --git a/en/resources/fine_tuning/index.md b/en/resources/fine_tuning/index.md index 4730288..97211d9 100644 --- a/en/resources/fine_tuning/index.md +++ b/en/resources/fine_tuning/index.md @@ -6,7 +6,7 @@ ### Dpo Hyperparameters -- `DpoHyperparameters = object { batch_size, beta, learning_rate_multiplier, n_epochs }` +- `DpoHyperparameters object { batch_size, beta, learning_rate_multiplier, n_epochs }` The hyperparameters used for the DPO fine-tuning job. @@ -52,7 +52,7 @@ ### Dpo Method -- `DpoMethod = object { hyperparameters }` +- `DpoMethod object { hyperparameters }` Configuration for the DPO fine-tuning method. @@ -102,7 +102,7 @@ ### Reinforcement Hyperparameters -- `ReinforcementHyperparameters = object { batch_size, compute_multiplier, eval_interval, 4 more }` +- `ReinforcementHyperparameters object { batch_size, compute_multiplier, eval_interval, 4 more }` The hyperparameters used for the reinforcement fine-tuning job. @@ -180,7 +180,7 @@ ### Reinforcement Method -- `ReinforcementMethod = object { grader, hyperparameters }` +- `ReinforcementMethod object { grader, hyperparameters }` Configuration for the reinforcement fine-tuning method. @@ -188,7 +188,7 @@ The grader used for the fine-tuning job. - - `StringCheckGrader = object { input, name, operation, 2 more }` + - `StringCheckGrader object { input, name, operation, 2 more }` A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. @@ -222,7 +222,7 @@ - `"string_check"` - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` A TextSimilarityGrader object which grades text based on similarity metrics. @@ -272,7 +272,7 @@ - `"text_similarity"` - - `PythonGrader = object { name, source, type, image_tag }` + - `PythonGrader object { name, source, type, image_tag }` A PythonGrader object that runs a python script on the input. @@ -294,7 +294,7 @@ The image tag to use for the python script. - - `ScoreModelGrader = object { input, model, name, 3 more }` + - `ScoreModelGrader object { input, model, name, 3 more }` A ScoreModelGrader object that uses a model to assign a score to the input. @@ -310,7 +310,7 @@ A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. @@ -324,7 +324,7 @@ - `"input_text"` - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -338,7 +338,7 @@ - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -356,7 +356,7 @@ The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. @@ -390,21 +390,11 @@ A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -418,7 +408,7 @@ - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -436,31 +426,10 @@ The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or @@ -543,7 +512,7 @@ An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `MultiGrader = object { calculate_output, graders, name, type }` + - `MultiGrader object { calculate_output, graders, name, type }` A MultiGrader object combines the output of multiple graders to produce a single score. @@ -555,1295 +524,1326 @@ A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - `StringCheckGrader = object { input, name, operation, 2 more }` + - `StringCheckGrader object { input, name, operation, 2 more }` A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - `input: string` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` - The input text. This may include template strings. + A TextSimilarityGrader object which grades text based on similarity metrics. - - `name: string` + - `PythonGrader object { name, source, type, image_tag }` - The name of the grader. + A PythonGrader object that runs a python script on the input. - - `operation: "eq" or "ne" or "like" or "ilike"` + - `ScoreModelGrader object { input, model, name, 3 more }` - The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. + A ScoreModelGrader object that uses a model to assign a score to the input. - - `"eq"` + - `LabelModelGrader object { input, labels, model, 3 more }` - - `"ne"` + A LabelModelGrader object which uses a model to assign labels to each item + in the evaluation. - - `"like"` + - `input: array of object { content, role, type }` - - `"ilike"` + - `content: string or ResponseInputText or object { text, type } or 3 more` - - `reference: string` + Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - The reference text. This may include template strings. + - `TextInput = string` - - `type: "string_check"` + A text input to the model. - The object type, which is always `string_check`. + - `ResponseInputText object { text, type }` - - `"string_check"` + A text input to the model. - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` + - `OutputText object { text, type }` - A TextSimilarityGrader object which grades text based on similarity metrics. + A text output from the model. - - `evaluation_metric: "cosine" or "fuzzy_match" or "bleu" or 8 more` + - `text: string` - The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, - `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, - or `rouge_l`. + The text output from the model. - - `"cosine"` + - `type: "output_text"` - - `"fuzzy_match"` + The type of the output text. Always `output_text`. - - `"bleu"` + - `"output_text"` - - `"gleu"` + - `InputImage object { image_url, type, detail }` - - `"meteor"` + An image input block used within EvalItem content arrays. - - `"rouge_1"` + - `image_url: string` - - `"rouge_2"` + The URL of the image input. - - `"rouge_3"` + - `type: "input_image"` - - `"rouge_4"` + The type of the image input. Always `input_image`. - - `"rouge_5"` + - `"input_image"` - - `"rouge_l"` + - `detail: optional string` - - `input: string` + The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - The text being graded. + - `ResponseInputAudio object { input_audio, type }` - - `name: string` + An audio input to the model. - The name of the grader. + - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - `reference: string` + A list of inputs, each of which may be either an input text, output text, input + image, or input audio object. - The text being graded against. + - `role: "user" or "assistant" or "system" or "developer"` - - `type: "text_similarity"` + The role of the message input. One of `user`, `assistant`, `system`, or + `developer`. - The type of grader. + - `"user"` - - `"text_similarity"` + - `"assistant"` - - `PythonGrader = object { name, source, type, image_tag }` + - `"system"` - A PythonGrader object that runs a python script on the input. + - `"developer"` - - `name: string` + - `type: optional "message"` - The name of the grader. + The type of the message input. Always `message`. - - `source: string` + - `"message"` - The source code of the python script. + - `labels: array of string` - - `type: "python"` + The labels to assign to each item in the evaluation. - The object type, which is always `python`. + - `model: string` - - `"python"` + The model to use for the evaluation. Must support structured outputs. - - `image_tag: optional string` + - `name: string` - The image tag to use for the python script. + The name of the grader. - - `ScoreModelGrader = object { input, model, name, 3 more }` + - `passing_labels: array of string` - A ScoreModelGrader object that uses a model to assign a score to the input. + The labels that indicate a passing result. Must be a subset of labels. - - `input: array of object { content, role, type }` + - `type: "label_model"` - The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. + The object type, which is always `label_model`. - - `content: string or ResponseInputText or object { text, type } or 3 more` + - `"label_model"` - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. + - `name: string` - - `TextInput = string` + The name of the grader. - A text input to the model. + - `type: "multi"` - - `ResponseInputText = object { text, type }` + The object type, which is always `multi`. - A text input to the model. + - `"multi"` - - `text: string` + - `hyperparameters: optional ReinforcementHyperparameters` - The text input to the model. + The hyperparameters used for the reinforcement fine-tuning job. - - `type: "input_text"` + - `batch_size: optional "auto" or number` - The type of the input item. Always `input_text`. + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - `"input_text"` + - `"auto"` - - `OutputText = object { text, type }` + - `"auto"` - A text output from the model. + - `number` - - `text: string` + - `compute_multiplier: optional "auto" or number` - The text output from the model. + Multiplier on amount of compute used for exploring search space during training. - - `type: "output_text"` + - `"auto"` - The type of the output text. Always `output_text`. + - `"auto"` - - `"output_text"` + - `number` - - `InputImage = object { image_url, type, detail }` + - `eval_interval: optional "auto" or number` - An image input block used within EvalItem content arrays. + The number of training steps between evaluation runs. - - `image_url: string` + - `"auto"` - The URL of the image input. + - `"auto"` - - `type: "input_image"` + - `number` - The type of the image input. Always `input_image`. + - `eval_samples: optional "auto" or number` - - `"input_image"` + Number of evaluation samples to generate per training step. - - `detail: optional string` + - `"auto"` - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + - `"auto"` - - `ResponseInputAudio = object { input_audio, type }` + - `number` - An audio input to the model. + - `learning_rate_multiplier: optional "auto" or number` - - `input_audio: object { data, format }` + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - `data: string` + - `"auto"` - Base64-encoded audio data. + - `"auto"` - - `format: "mp3" or "wav"` + - `number` - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + - `n_epochs: optional "auto" or number` - - `"mp3"` + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - `"wav"` + - `"auto"` - - `type: "input_audio"` + - `"auto"` - The type of the input item. Always `input_audio`. + - `number` - - `"input_audio"` + - `reasoning_effort: optional "default" or "low" or "medium" or "high"` - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` + Level of reasoning effort. - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. + - `"default"` - - `TextInput = string` + - `"low"` - A text input to the model. + - `"medium"` - - `ResponseInputText = object { text, type }` + - `"high"` - A text input to the model. +### Supervised Hyperparameters - - `text: string` +- `SupervisedHyperparameters object { batch_size, learning_rate_multiplier, n_epochs }` - The text input to the model. + The hyperparameters used for the fine-tuning job. - - `type: "input_text"` + - `batch_size: optional "auto" or number` - The type of the input item. Always `input_text`. + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - `"input_text"` + - `"auto"` - - `OutputText = object { text, type }` + - `"auto"` - A text output from the model. + - `number` - - `text: string` + - `learning_rate_multiplier: optional "auto" or number` - The text output from the model. + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - `type: "output_text"` + - `"auto"` - The type of the output text. Always `output_text`. + - `"auto"` - - `"output_text"` + - `number` - - `InputImage = object { image_url, type, detail }` + - `n_epochs: optional "auto" or number` - An image input block used within EvalItem content arrays. + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - `image_url: string` + - `"auto"` - The URL of the image input. + - `"auto"` - - `type: "input_image"` + - `number` - The type of the image input. Always `input_image`. +### Supervised Method - - `"input_image"` +- `SupervisedMethod object { hyperparameters }` - - `detail: optional string` + Configuration for the supervised fine-tuning method. - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + - `hyperparameters: optional SupervisedHyperparameters` - - `ResponseInputAudio = object { input_audio, type }` + The hyperparameters used for the fine-tuning job. - An audio input to the model. + - `batch_size: optional "auto" or number` - - `input_audio: object { data, format }` + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - `data: string` + - `"auto"` - Base64-encoded audio data. + - `"auto"` - - `format: "mp3" or "wav"` + - `number` - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + - `learning_rate_multiplier: optional "auto" or number` - - `"mp3"` + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - `"wav"` + - `"auto"` - - `type: "input_audio"` + - `"auto"` - The type of the input item. Always `input_audio`. + - `number` - - `"input_audio"` + - `n_epochs: optional "auto" or number` - - `role: "user" or "assistant" or "system" or "developer"` + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. + - `"auto"` - - `"user"` + - `"auto"` - - `"assistant"` + - `number` - - `"system"` +# Jobs - - `"developer"` +## Create fine-tuning job - - `type: optional "message"` +**post** `/fine_tuning/jobs` - The type of the message input. Always `message`. +Creates a fine-tuning job which begins the process of creating a new model from a given dataset. - - `"message"` +Response includes details of the enqueued job including job status and the name of the fine-tuned models once complete. - - `model: string` +[Learn more about fine-tuning](/docs/guides/model-optimization) - The model to use for the evaluation. +### Body Parameters - - `name: string` +- `model: string or "babbage-002" or "davinci-002" or "gpt-3.5-turbo" or "gpt-4o-mini"` - The name of the grader. + The name of the model to fine-tune. You can select one of the + [supported models](/docs/guides/fine-tuning#which-models-can-be-fine-tuned). - - `type: "score_model"` + - `string` - The object type, which is always `score_model`. + - `"babbage-002" or "davinci-002" or "gpt-3.5-turbo" or "gpt-4o-mini"` - - `"score_model"` + The name of the model to fine-tune. You can select one of the + [supported models](/docs/guides/fine-tuning#which-models-can-be-fine-tuned). - - `range: optional array of number` + - `"babbage-002"` - The range of the score. Defaults to `[0, 1]`. + - `"davinci-002"` - - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` + - `"gpt-3.5-turbo"` - The sampling parameters for the model. + - `"gpt-4o-mini"` - - `max_completions_tokens: optional number` +- `training_file: string` - The maximum number of tokens the grader model may generate in its response. + The ID of an uploaded file that contains training data. - - `reasoning_effort: optional ReasoningEffort` + See [upload file](/docs/api-reference/files/create) for how to upload a file. - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. + Your dataset must be formatted as a JSONL file. Additionally, you must upload your file with the purpose `fine-tune`. - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + The contents of the file should differ depending on if the model uses the [chat](/docs/api-reference/fine-tuning/chat-input), [completions](/docs/api-reference/fine-tuning/completions-input) format, or if the fine-tuning method uses the [preference](/docs/api-reference/fine-tuning/preference-input) format. - - `"none"` + See the [fine-tuning guide](/docs/guides/model-optimization) for more details. - - `"minimal"` +- `hyperparameters: optional object { batch_size, learning_rate_multiplier, n_epochs }` - - `"low"` + The hyperparameters used for the fine-tuning job. + This value is now deprecated in favor of `method`, and should be passed in under the `method` parameter. - - `"medium"` + - `batch_size: optional "auto" or number` - - `"high"` + Number of examples in each batch. A larger batch size means that model parameters + are updated less frequently, but with lower variance. - - `"xhigh"` + - `"auto"` - - `seed: optional number` + - `"auto"` - A seed value to initialize the randomness, during sampling. + - `number` - - `temperature: optional number` + - `learning_rate_multiplier: optional "auto" or number` - A higher temperature increases randomness in the outputs. + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid + overfitting. - - `top_p: optional number` + - `"auto"` - An alternative to temperature for nucleus sampling; 1.0 includes all tokens. + - `"auto"` - - `LabelModelGrader = object { input, labels, model, 3 more }` + - `number` - A LabelModelGrader object which uses a model to assign labels to each item - in the evaluation. + - `n_epochs: optional "auto" or number` - - `input: array of object { content, role, type }` + The number of epochs to train the model for. An epoch refers to one full cycle + through the training dataset. - - `content: string or ResponseInputText or object { text, type } or 3 more` + - `"auto"` - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. + - `"auto"` - - `TextInput = string` + - `number` - A text input to the model. +- `integrations: optional array of object { type, wandb }` - - `ResponseInputText = object { text, type }` + A list of integrations to enable for your fine-tuning job. - A text input to the model. + - `type: "wandb"` - - `text: string` + The type of integration to enable. Currently, only "wandb" (Weights and Biases) is supported. - The text input to the model. + - `"wandb"` - - `type: "input_text"` + - `wandb: object { project, entity, name, tags }` - The type of the input item. Always `input_text`. + The settings for your integration with Weights and Biases. This payload specifies the project that + metrics will be sent to. Optionally, you can set an explicit display name for your run, add tags + to your run, and set a default entity (team, username, etc) to be associated with your run. - - `"input_text"` + - `project: string` - - `OutputText = object { text, type }` + The name of the project that the new run will be created under. - A text output from the model. + - `entity: optional string` - - `text: string` + The entity to use for the run. This allows you to set the team or username of the WandB user that you would + like associated with the run. If not set, the default entity for the registered WandB API key is used. - The text output from the model. + - `name: optional string` - - `type: "output_text"` + A display name to set for the run. If not set, we will use the Job ID as the name. - The type of the output text. Always `output_text`. + - `tags: optional array of string` - - `"output_text"` + A list of tags to be attached to the newly created run. These tags are passed through directly to WandB. Some + default tags are generated by OpenAI: "openai/finetune", "openai/{base-model}", "openai/{ftjob-abcdef}". - - `InputImage = object { image_url, type, detail }` +- `metadata: optional Metadata` - An image input block used within EvalItem content arrays. + Set of 16 key-value pairs that can be attached to an object. This can be + useful for storing additional information about the object in a structured + format, and querying for objects via API or the dashboard. - - `image_url: string` + Keys are strings with a maximum length of 64 characters. Values are strings + with a maximum length of 512 characters. - The URL of the image input. +- `method: optional object { type, dpo, reinforcement, supervised }` - - `type: "input_image"` + The method used for fine-tuning. - The type of the image input. Always `input_image`. + - `type: "supervised" or "dpo" or "reinforcement"` - - `"input_image"` + The type of method. Is either `supervised`, `dpo`, or `reinforcement`. - - `detail: optional string` + - `"supervised"` - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + - `"dpo"` - - `ResponseInputAudio = object { input_audio, type }` + - `"reinforcement"` - An audio input to the model. + - `dpo: optional DpoMethod` - - `input_audio: object { data, format }` + Configuration for the DPO fine-tuning method. - - `data: string` + - `hyperparameters: optional DpoHyperparameters` - Base64-encoded audio data. + The hyperparameters used for the DPO fine-tuning job. - - `format: "mp3" or "wav"` + - `batch_size: optional "auto" or number` - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - `"mp3"` + - `"auto"` - - `"wav"` + - `"auto"` - - `type: "input_audio"` + - `number` - The type of the input item. Always `input_audio`. + - `beta: optional "auto" or number` - - `"input_audio"` + The beta value for the DPO method. A higher beta value will increase the weight of the penalty between the policy and reference model. - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` + - `"auto"` - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. + - `"auto"` - - `TextInput = string` + - `number` - A text input to the model. + - `learning_rate_multiplier: optional "auto" or number` - - `ResponseInputText = object { text, type }` + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - A text input to the model. + - `"auto"` - - `text: string` + - `"auto"` - The text input to the model. + - `number` - - `type: "input_text"` + - `n_epochs: optional "auto" or number` - The type of the input item. Always `input_text`. + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - `"input_text"` + - `"auto"` - - `OutputText = object { text, type }` + - `"auto"` - A text output from the model. + - `number` - - `text: string` + - `reinforcement: optional ReinforcementMethod` - The text output from the model. + Configuration for the reinforcement fine-tuning method. - - `type: "output_text"` + - `grader: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - The type of the output text. Always `output_text`. + The grader used for the fine-tuning job. - - `"output_text"` + - `StringCheckGrader object { input, name, operation, 2 more }` - - `InputImage = object { image_url, type, detail }` + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - An image input block used within EvalItem content arrays. + - `input: string` - - `image_url: string` + The input text. This may include template strings. - The URL of the image input. + - `name: string` - - `type: "input_image"` + The name of the grader. - The type of the image input. Always `input_image`. + - `operation: "eq" or "ne" or "like" or "ilike"` - - `"input_image"` + The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. - - `detail: optional string` + - `"eq"` - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + - `"ne"` - - `ResponseInputAudio = object { input_audio, type }` + - `"like"` - An audio input to the model. + - `"ilike"` - - `input_audio: object { data, format }` - - - `data: string` + - `reference: string` - Base64-encoded audio data. + The reference text. This may include template strings. - - `format: "mp3" or "wav"` + - `type: "string_check"` - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + The object type, which is always `string_check`. - - `"mp3"` + - `"string_check"` - - `"wav"` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` - - `type: "input_audio"` + A TextSimilarityGrader object which grades text based on similarity metrics. - The type of the input item. Always `input_audio`. + - `evaluation_metric: "cosine" or "fuzzy_match" or "bleu" or 8 more` - - `"input_audio"` + The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, + `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, + or `rouge_l`. - - `role: "user" or "assistant" or "system" or "developer"` + - `"cosine"` - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. + - `"fuzzy_match"` - - `"user"` + - `"bleu"` - - `"assistant"` + - `"gleu"` - - `"system"` + - `"meteor"` - - `"developer"` + - `"rouge_1"` - - `type: optional "message"` + - `"rouge_2"` - The type of the message input. Always `message`. + - `"rouge_3"` - - `"message"` + - `"rouge_4"` - - `labels: array of string` + - `"rouge_5"` - The labels to assign to each item in the evaluation. + - `"rouge_l"` - - `model: string` + - `input: string` - The model to use for the evaluation. Must support structured outputs. + The text being graded. - `name: string` The name of the grader. - - `passing_labels: array of string` + - `reference: string` - The labels that indicate a passing result. Must be a subset of labels. + The text being graded against. - - `type: "label_model"` + - `type: "text_similarity"` - The object type, which is always `label_model`. + The type of grader. - - `"label_model"` + - `"text_similarity"` + + - `PythonGrader object { name, source, type, image_tag }` + + A PythonGrader object that runs a python script on the input. - `name: string` The name of the grader. - - `type: "multi"` + - `source: string` - The object type, which is always `multi`. + The source code of the python script. - - `"multi"` + - `type: "python"` - - `hyperparameters: optional ReinforcementHyperparameters` + The object type, which is always `python`. - The hyperparameters used for the reinforcement fine-tuning job. + - `"python"` - - `batch_size: optional "auto" or number` + - `image_tag: optional string` - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. + The image tag to use for the python script. - - `"auto"` + - `ScoreModelGrader object { input, model, name, 3 more }` - - `"auto"` + A ScoreModelGrader object that uses a model to assign a score to the input. - - `number` + - `input: array of object { content, role, type }` - - `compute_multiplier: optional "auto" or number` + The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. - Multiplier on amount of compute used for exploring search space during training. + - `content: string or ResponseInputText or object { text, type } or 3 more` - - `"auto"` + Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - - `"auto"` + - `TextInput = string` - - `number` + A text input to the model. - - `eval_interval: optional "auto" or number` + - `ResponseInputText object { text, type }` - The number of training steps between evaluation runs. + A text input to the model. - - `"auto"` + - `text: string` - - `"auto"` + The text input to the model. - - `number` + - `type: "input_text"` - - `eval_samples: optional "auto" or number` + The type of the input item. Always `input_text`. - Number of evaluation samples to generate per training step. + - `"input_text"` - - `"auto"` + - `OutputText object { text, type }` - - `"auto"` + A text output from the model. - - `number` + - `text: string` - - `learning_rate_multiplier: optional "auto" or number` + The text output from the model. - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. + - `type: "output_text"` - - `"auto"` + The type of the output text. Always `output_text`. - - `"auto"` + - `"output_text"` - - `number` + - `InputImage object { image_url, type, detail }` - - `n_epochs: optional "auto" or number` + An image input block used within EvalItem content arrays. - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. + - `image_url: string` - - `"auto"` + The URL of the image input. - - `"auto"` + - `type: "input_image"` - - `number` + The type of the image input. Always `input_image`. - - `reasoning_effort: optional "default" or "low" or "medium" or "high"` + - `"input_image"` - Level of reasoning effort. + - `detail: optional string` - - `"default"` + The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `"low"` + - `ResponseInputAudio object { input_audio, type }` - - `"medium"` + An audio input to the model. - - `"high"` + - `input_audio: object { data, format }` -### Supervised Hyperparameters + - `data: string` -- `SupervisedHyperparameters = object { batch_size, learning_rate_multiplier, n_epochs }` + Base64-encoded audio data. - The hyperparameters used for the fine-tuning job. + - `format: "mp3" or "wav"` - - `batch_size: optional "auto" or number` + The format of the audio data. Currently supported formats are `mp3` and + `wav`. - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. + - `"mp3"` - - `"auto"` + - `"wav"` - - `"auto"` + - `type: "input_audio"` - - `number` + The type of the input item. Always `input_audio`. - - `learning_rate_multiplier: optional "auto" or number` + - `"input_audio"` - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. + - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - `"auto"` + A list of inputs, each of which may be either an input text, output text, input + image, or input audio object. - - `"auto"` + - `TextInput = string` - - `number` + A text input to the model. - - `n_epochs: optional "auto" or number` + - `ResponseInputText object { text, type }` - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. + A text input to the model. - - `"auto"` + - `OutputText object { text, type }` - - `"auto"` + A text output from the model. - - `number` + - `text: string` -### Supervised Method + The text output from the model. -- `SupervisedMethod = object { hyperparameters }` + - `type: "output_text"` - Configuration for the supervised fine-tuning method. + The type of the output text. Always `output_text`. - - `hyperparameters: optional SupervisedHyperparameters` + - `"output_text"` - The hyperparameters used for the fine-tuning job. + - `InputImage object { image_url, type, detail }` - - `batch_size: optional "auto" or number` + An image input block used within EvalItem content arrays. - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. + - `image_url: string` - - `"auto"` + The URL of the image input. - - `"auto"` + - `type: "input_image"` - - `number` + The type of the image input. Always `input_image`. - - `learning_rate_multiplier: optional "auto" or number` + - `"input_image"` - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. + - `detail: optional string` - - `"auto"` + The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `"auto"` + - `ResponseInputAudio object { input_audio, type }` - - `number` + An audio input to the model. - - `n_epochs: optional "auto" or number` + - `role: "user" or "assistant" or "system" or "developer"` - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. + The role of the message input. One of `user`, `assistant`, `system`, or + `developer`. - - `"auto"` + - `"user"` - - `"auto"` + - `"assistant"` - - `number` + - `"system"` -# Jobs + - `"developer"` -## Create fine-tuning job + - `type: optional "message"` -**post** `/fine_tuning/jobs` + The type of the message input. Always `message`. -Creates a fine-tuning job which begins the process of creating a new model from a given dataset. + - `"message"` -Response includes details of the enqueued job including job status and the name of the fine-tuned models once complete. + - `model: string` -[Learn more about fine-tuning](/docs/guides/model-optimization) + The model to use for the evaluation. -### Body Parameters + - `name: string` -- `model: string or "babbage-002" or "davinci-002" or "gpt-3.5-turbo" or "gpt-4o-mini"` + The name of the grader. - The name of the model to fine-tune. You can select one of the - [supported models](/docs/guides/fine-tuning#which-models-can-be-fine-tuned). + - `type: "score_model"` - - `string` + The object type, which is always `score_model`. - - `"babbage-002" or "davinci-002" or "gpt-3.5-turbo" or "gpt-4o-mini"` + - `"score_model"` - The name of the model to fine-tune. You can select one of the - [supported models](/docs/guides/fine-tuning#which-models-can-be-fine-tuned). + - `range: optional array of number` - - `"babbage-002"` + The range of the score. Defaults to `[0, 1]`. - - `"davinci-002"` + - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` - - `"gpt-3.5-turbo"` + The sampling parameters for the model. - - `"gpt-4o-mini"` + - `max_completions_tokens: optional number` -- `training_file: string` + The maximum number of tokens the grader model may generate in its response. - The ID of an uploaded file that contains training data. + - `reasoning_effort: optional ReasoningEffort` - See [upload file](/docs/api-reference/files/create) for how to upload a file. + Constrains effort on reasoning for + [reasoning models](https://platform.openai.com/docs/guides/reasoning). + Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing + reasoning effort can result in faster responses and fewer tokens used + on reasoning in a response. - Your dataset must be formatted as a JSONL file. Additionally, you must upload your file with the purpose `fine-tune`. - - The contents of the file should differ depending on if the model uses the [chat](/docs/api-reference/fine-tuning/chat-input), [completions](/docs/api-reference/fine-tuning/completions-input) format, or if the fine-tuning method uses the [preference](/docs/api-reference/fine-tuning/preference-input) format. + - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. + - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. + - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. + - `xhigh` is supported for all models after `gpt-5.1-codex-max`. - See the [fine-tuning guide](/docs/guides/model-optimization) for more details. + - `"none"` -- `hyperparameters: optional object { batch_size, learning_rate_multiplier, n_epochs }` + - `"minimal"` - The hyperparameters used for the fine-tuning job. - This value is now deprecated in favor of `method`, and should be passed in under the `method` parameter. + - `"low"` - - `batch_size: optional "auto" or number` + - `"medium"` - Number of examples in each batch. A larger batch size means that model parameters - are updated less frequently, but with lower variance. + - `"high"` - - `"auto"` + - `"xhigh"` - - `"auto"` + - `seed: optional number` - - `number` + A seed value to initialize the randomness, during sampling. - - `learning_rate_multiplier: optional "auto" or number` + - `temperature: optional number` - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid - overfitting. + A higher temperature increases randomness in the outputs. - - `"auto"` + - `top_p: optional number` - - `"auto"` + An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `number` + - `MultiGrader object { calculate_output, graders, name, type }` - - `n_epochs: optional "auto" or number` + A MultiGrader object combines the output of multiple graders to produce a single score. - The number of epochs to train the model for. An epoch refers to one full cycle - through the training dataset. + - `calculate_output: string` - - `"auto"` + A formula to calculate the output based on grader results. - - `"auto"` + - `graders: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - - `number` + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. -- `integrations: optional array of object { type, wandb }` + - `StringCheckGrader object { input, name, operation, 2 more }` - A list of integrations to enable for your fine-tuning job. + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - `type: "wandb"` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` - The type of integration to enable. Currently, only "wandb" (Weights and Biases) is supported. + A TextSimilarityGrader object which grades text based on similarity metrics. - - `"wandb"` + - `PythonGrader object { name, source, type, image_tag }` - - `wandb: object { project, entity, name, tags }` + A PythonGrader object that runs a python script on the input. - The settings for your integration with Weights and Biases. This payload specifies the project that - metrics will be sent to. Optionally, you can set an explicit display name for your run, add tags - to your run, and set a default entity (team, username, etc) to be associated with your run. + - `ScoreModelGrader object { input, model, name, 3 more }` - - `project: string` + A ScoreModelGrader object that uses a model to assign a score to the input. - The name of the project that the new run will be created under. + - `LabelModelGrader object { input, labels, model, 3 more }` - - `entity: optional string` + A LabelModelGrader object which uses a model to assign labels to each item + in the evaluation. - The entity to use for the run. This allows you to set the team or username of the WandB user that you would - like associated with the run. If not set, the default entity for the registered WandB API key is used. + - `input: array of object { content, role, type }` - - `name: optional string` + - `content: string or ResponseInputText or object { text, type } or 3 more` - A display name to set for the run. If not set, we will use the Job ID as the name. + Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - - `tags: optional array of string` + - `TextInput = string` - A list of tags to be attached to the newly created run. These tags are passed through directly to WandB. Some - default tags are generated by OpenAI: "openai/finetune", "openai/{base-model}", "openai/{ftjob-abcdef}". + A text input to the model. -- `metadata: optional Metadata` + - `ResponseInputText object { text, type }` - Set of 16 key-value pairs that can be attached to an object. This can be - useful for storing additional information about the object in a structured - format, and querying for objects via API or the dashboard. + A text input to the model. - Keys are strings with a maximum length of 64 characters. Values are strings - with a maximum length of 512 characters. + - `OutputText object { text, type }` -- `method: optional object { type, dpo, reinforcement, supervised }` + A text output from the model. - The method used for fine-tuning. + - `text: string` - - `type: "supervised" or "dpo" or "reinforcement"` + The text output from the model. - The type of method. Is either `supervised`, `dpo`, or `reinforcement`. + - `type: "output_text"` - - `"supervised"` + The type of the output text. Always `output_text`. - - `"dpo"` + - `"output_text"` - - `"reinforcement"` + - `InputImage object { image_url, type, detail }` - - `dpo: optional DpoMethod` + An image input block used within EvalItem content arrays. - Configuration for the DPO fine-tuning method. + - `image_url: string` - - `hyperparameters: optional DpoHyperparameters` + The URL of the image input. - The hyperparameters used for the DPO fine-tuning job. + - `type: "input_image"` - - `batch_size: optional "auto" or number` + The type of the image input. Always `input_image`. - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. + - `"input_image"` - - `"auto"` + - `detail: optional string` - - `"auto"` + The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `number` + - `ResponseInputAudio object { input_audio, type }` - - `beta: optional "auto" or number` + An audio input to the model. - The beta value for the DPO method. A higher beta value will increase the weight of the penalty between the policy and reference model. + - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - `"auto"` + A list of inputs, each of which may be either an input text, output text, input + image, or input audio object. - - `"auto"` + - `role: "user" or "assistant" or "system" or "developer"` - - `number` + The role of the message input. One of `user`, `assistant`, `system`, or + `developer`. - - `learning_rate_multiplier: optional "auto" or number` + - `"user"` - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. + - `"assistant"` - - `"auto"` + - `"system"` - - `"auto"` + - `"developer"` - - `number` + - `type: optional "message"` - - `n_epochs: optional "auto" or number` + The type of the message input. Always `message`. - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. + - `"message"` - - `"auto"` + - `labels: array of string` - - `"auto"` + The labels to assign to each item in the evaluation. - - `number` + - `model: string` - - `reinforcement: optional ReinforcementMethod` + The model to use for the evaluation. Must support structured outputs. - Configuration for the reinforcement fine-tuning method. + - `name: string` - - `grader: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` + The name of the grader. - The grader used for the fine-tuning job. + - `passing_labels: array of string` - - `StringCheckGrader = object { input, name, operation, 2 more }` + The labels that indicate a passing result. Must be a subset of labels. - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. + - `type: "label_model"` - - `input: string` + The object type, which is always `label_model`. - The input text. This may include template strings. + - `"label_model"` - `name: string` The name of the grader. - - `operation: "eq" or "ne" or "like" or "ilike"` + - `type: "multi"` - The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. + The object type, which is always `multi`. - - `"eq"` + - `"multi"` - - `"ne"` + - `hyperparameters: optional ReinforcementHyperparameters` - - `"like"` + The hyperparameters used for the reinforcement fine-tuning job. - - `"ilike"` + - `batch_size: optional "auto" or number` - - `reference: string` + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - The reference text. This may include template strings. + - `"auto"` - - `type: "string_check"` + - `"auto"` - The object type, which is always `string_check`. + - `number` - - `"string_check"` + - `compute_multiplier: optional "auto" or number` - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` + Multiplier on amount of compute used for exploring search space during training. - A TextSimilarityGrader object which grades text based on similarity metrics. + - `"auto"` - - `evaluation_metric: "cosine" or "fuzzy_match" or "bleu" or 8 more` + - `"auto"` - The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, - `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, - or `rouge_l`. + - `number` - - `"cosine"` + - `eval_interval: optional "auto" or number` - - `"fuzzy_match"` + The number of training steps between evaluation runs. - - `"bleu"` + - `"auto"` - - `"gleu"` + - `"auto"` - - `"meteor"` + - `number` - - `"rouge_1"` + - `eval_samples: optional "auto" or number` - - `"rouge_2"` + Number of evaluation samples to generate per training step. - - `"rouge_3"` + - `"auto"` - - `"rouge_4"` + - `"auto"` - - `"rouge_5"` + - `number` - - `"rouge_l"` + - `learning_rate_multiplier: optional "auto" or number` - - `input: string` + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - The text being graded. + - `"auto"` - - `name: string` + - `"auto"` - The name of the grader. + - `number` - - `reference: string` + - `n_epochs: optional "auto" or number` - The text being graded against. + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - `type: "text_similarity"` + - `"auto"` - The type of grader. + - `"auto"` - - `"text_similarity"` + - `number` - - `PythonGrader = object { name, source, type, image_tag }` + - `reasoning_effort: optional "default" or "low" or "medium" or "high"` - A PythonGrader object that runs a python script on the input. + Level of reasoning effort. - - `name: string` + - `"default"` - The name of the grader. + - `"low"` - - `source: string` + - `"medium"` - The source code of the python script. + - `"high"` - - `type: "python"` + - `supervised: optional SupervisedMethod` - The object type, which is always `python`. + Configuration for the supervised fine-tuning method. - - `"python"` + - `hyperparameters: optional SupervisedHyperparameters` - - `image_tag: optional string` + The hyperparameters used for the fine-tuning job. - The image tag to use for the python script. + - `batch_size: optional "auto" or number` - - `ScoreModelGrader = object { input, model, name, 3 more }` + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - A ScoreModelGrader object that uses a model to assign a score to the input. + - `"auto"` - - `input: array of object { content, role, type }` + - `"auto"` - The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. + - `number` - - `content: string or ResponseInputText or object { text, type } or 3 more` + - `learning_rate_multiplier: optional "auto" or number` - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - `TextInput = string` + - `"auto"` - A text input to the model. + - `"auto"` - - `ResponseInputText = object { text, type }` + - `number` - A text input to the model. + - `n_epochs: optional "auto" or number` - - `text: string` + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - The text input to the model. + - `"auto"` - - `type: "input_text"` + - `"auto"` - The type of the input item. Always `input_text`. + - `number` - - `"input_text"` +- `seed: optional number` - - `OutputText = object { text, type }` + The seed controls the reproducibility of the job. Passing in the same seed and job parameters should produce the same results, but may differ in rare cases. + If a seed is not specified, one will be generated for you. - A text output from the model. +- `suffix: optional string` - - `text: string` + A string of up to 64 characters that will be added to your fine-tuned model name. - The text output from the model. + For example, a `suffix` of "custom-model-name" would produce a model name like `ft:gpt-4o-mini:openai:custom-model-name:7p4lURel`. - - `type: "output_text"` +- `validation_file: optional string` - The type of the output text. Always `output_text`. + The ID of an uploaded file that contains validation data. - - `"output_text"` + If you provide this file, the data is used to generate validation + metrics periodically during fine-tuning. These metrics can be viewed in + the fine-tuning results file. + The same data should not be present in both train and validation files. - - `InputImage = object { image_url, type, detail }` + Your dataset must be formatted as a JSONL file. You must upload your file with the purpose `fine-tune`. - An image input block used within EvalItem content arrays. + See the [fine-tuning guide](/docs/guides/model-optimization) for more details. - - `image_url: string` +### Returns - The URL of the image input. +- `FineTuningJob object { id, created_at, error, 16 more }` - - `type: "input_image"` + The `fine_tuning.job` object represents a fine-tuning job that has been created through the API. - The type of the image input. Always `input_image`. + - `id: string` - - `"input_image"` + The object identifier, which can be referenced in the API endpoints. - - `detail: optional string` + - `created_at: number` - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + The Unix timestamp (in seconds) for when the fine-tuning job was created. - - `ResponseInputAudio = object { input_audio, type }` + - `error: object { code, message, param }` - An audio input to the model. + For fine-tuning jobs that have `failed`, this will contain more information on the cause of the failure. - - `input_audio: object { data, format }` + - `code: string` - - `data: string` + A machine-readable error code. - Base64-encoded audio data. + - `message: string` - - `format: "mp3" or "wav"` + A human-readable error message. - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + - `param: string` - - `"mp3"` + The parameter that was invalid, usually `training_file` or `validation_file`. This field will be null if the failure was not parameter-specific. - - `"wav"` + - `fine_tuned_model: string` - - `type: "input_audio"` + The name of the fine-tuned model that is being created. The value will be null if the fine-tuning job is still running. - The type of the input item. Always `input_audio`. + - `finished_at: number` - - `"input_audio"` + The Unix timestamp (in seconds) for when the fine-tuning job was finished. The value will be null if the fine-tuning job is still running. - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` + - `hyperparameters: object { batch_size, learning_rate_multiplier, n_epochs }` - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. + The hyperparameters used for the fine-tuning job. This value will only be returned when running `supervised` jobs. - - `TextInput = string` + - `batch_size: optional "auto" or number` - A text input to the model. + Number of examples in each batch. A larger batch size means that model parameters + are updated less frequently, but with lower variance. - - `ResponseInputText = object { text, type }` + - `"auto"` - A text input to the model. + - `"auto"` - - `text: string` + - `number` - The text input to the model. + - `learning_rate_multiplier: optional "auto" or number` - - `type: "input_text"` + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid + overfitting. - The type of the input item. Always `input_text`. + - `"auto"` - - `"input_text"` + - `"auto"` - - `OutputText = object { text, type }` + - `number` - A text output from the model. + - `n_epochs: optional "auto" or number` - - `text: string` + The number of epochs to train the model for. An epoch refers to one full cycle + through the training dataset. - The text output from the model. + - `"auto"` - - `type: "output_text"` + - `"auto"` - The type of the output text. Always `output_text`. + - `number` - - `"output_text"` + - `model: string` - - `InputImage = object { image_url, type, detail }` + The base model that is being fine-tuned. - An image input block used within EvalItem content arrays. + - `object: "fine_tuning.job"` - - `image_url: string` + The object type, which is always "fine_tuning.job". - The URL of the image input. + - `"fine_tuning.job"` - - `type: "input_image"` + - `organization_id: string` - The type of the image input. Always `input_image`. + The organization that owns the fine-tuning job. - - `"input_image"` + - `result_files: array of string` - - `detail: optional string` + The compiled results file ID(s) for the fine-tuning job. You can retrieve the results with the [Files API](/docs/api-reference/files/retrieve-contents). - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + - `seed: number` - - `ResponseInputAudio = object { input_audio, type }` + The seed used for the fine-tuning job. - An audio input to the model. + - `status: "validating_files" or "queued" or "running" or 3 more` - - `input_audio: object { data, format }` + The current status of the fine-tuning job, which can be either `validating_files`, `queued`, `running`, `succeeded`, `failed`, or `cancelled`. - - `data: string` + - `"validating_files"` - Base64-encoded audio data. + - `"queued"` - - `format: "mp3" or "wav"` + - `"running"` - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + - `"succeeded"` - - `"mp3"` + - `"failed"` - - `"wav"` + - `"cancelled"` - - `type: "input_audio"` + - `trained_tokens: number` - The type of the input item. Always `input_audio`. + The total number of billable tokens processed by this fine-tuning job. The value will be null if the fine-tuning job is still running. - - `"input_audio"` + - `training_file: string` - - `role: "user" or "assistant" or "system" or "developer"` + The file ID used for training. You can retrieve the training data with the [Files API](/docs/api-reference/files/retrieve-contents). - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. + - `validation_file: string` - - `"user"` + The file ID used for validation. You can retrieve the validation results with the [Files API](/docs/api-reference/files/retrieve-contents). - - `"assistant"` + - `estimated_finish: optional number` - - `"system"` + The Unix timestamp (in seconds) for when the fine-tuning job is estimated to finish. The value will be null if the fine-tuning job is not running. - - `"developer"` + - `integrations: optional array of FineTuningJobWandbIntegrationObject` - - `type: optional "message"` + A list of integrations to enable for this fine-tuning job. - The type of the message input. Always `message`. + - `type: "wandb"` - - `"message"` + The type of the integration being enabled for the fine-tuning job - - `model: string` + - `"wandb"` - The model to use for the evaluation. + - `wandb: FineTuningJobWandbIntegration` - - `name: string` + The settings for your integration with Weights and Biases. This payload specifies the project that + metrics will be sent to. Optionally, you can set an explicit display name for your run, add tags + to your run, and set a default entity (team, username, etc) to be associated with your run. - The name of the grader. + - `project: string` - - `type: "score_model"` + The name of the project that the new run will be created under. - The object type, which is always `score_model`. + - `entity: optional string` - - `"score_model"` + The entity to use for the run. This allows you to set the team or username of the WandB user that you would + like associated with the run. If not set, the default entity for the registered WandB API key is used. - - `range: optional array of number` + - `name: optional string` - The range of the score. Defaults to `[0, 1]`. + A display name to set for the run. If not set, we will use the Job ID as the name. - - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` + - `tags: optional array of string` - The sampling parameters for the model. + A list of tags to be attached to the newly created run. These tags are passed through directly to WandB. Some + default tags are generated by OpenAI: "openai/finetune", "openai/{base-model}", "openai/{ftjob-abcdef}". - - `max_completions_tokens: optional number` + - `metadata: optional Metadata` - The maximum number of tokens the grader model may generate in its response. + Set of 16 key-value pairs that can be attached to an object. This can be + useful for storing additional information about the object in a structured + format, and querying for objects via API or the dashboard. - - `reasoning_effort: optional ReasoningEffort` + Keys are strings with a maximum length of 64 characters. Values are strings + with a maximum length of 512 characters. - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. + - `method: optional object { type, dpo, reinforcement, supervised }` - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + The method used for fine-tuning. - - `"none"` + - `type: "supervised" or "dpo" or "reinforcement"` - - `"minimal"` + The type of method. Is either `supervised`, `dpo`, or `reinforcement`. - - `"low"` + - `"supervised"` - - `"medium"` + - `"dpo"` - - `"high"` + - `"reinforcement"` - - `"xhigh"` + - `dpo: optional DpoMethod` - - `seed: optional number` + Configuration for the DPO fine-tuning method. - A seed value to initialize the randomness, during sampling. + - `hyperparameters: optional DpoHyperparameters` - - `temperature: optional number` + The hyperparameters used for the DPO fine-tuning job. - A higher temperature increases randomness in the outputs. + - `batch_size: optional "auto" or number` - - `top_p: optional number` + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - An alternative to temperature for nucleus sampling; 1.0 includes all tokens. + - `"auto"` - - `MultiGrader = object { calculate_output, graders, name, type }` + - `"auto"` - A MultiGrader object combines the output of multiple graders to produce a single score. + - `number` - - `calculate_output: string` + - `beta: optional "auto" or number` - A formula to calculate the output based on grader results. + The beta value for the DPO method. A higher beta value will increase the weight of the penalty between the policy and reference model. - - `graders: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` + - `"auto"` - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. + - `"auto"` + + - `number` + + - `learning_rate_multiplier: optional "auto" or number` + + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. + + - `"auto"` + + - `"auto"` + + - `number` + + - `n_epochs: optional "auto" or number` + + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. + + - `"auto"` + + - `"auto"` + + - `number` + + - `reinforcement: optional ReinforcementMethod` + + Configuration for the reinforcement fine-tuning method. + + - `grader: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - - `StringCheckGrader = object { input, name, operation, 2 more }` + The grader used for the fine-tuning job. + + - `StringCheckGrader object { input, name, operation, 2 more }` A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. @@ -1877,7 +1877,7 @@ Response includes details of the enqueued job including job status and the name - `"string_check"` - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` A TextSimilarityGrader object which grades text based on similarity metrics. @@ -1927,7 +1927,7 @@ Response includes details of the enqueued job including job status and the name - `"text_similarity"` - - `PythonGrader = object { name, source, type, image_tag }` + - `PythonGrader object { name, source, type, image_tag }` A PythonGrader object that runs a python script on the input. @@ -1949,7 +1949,7 @@ Response includes details of the enqueued job including job status and the name The image tag to use for the python script. - - `ScoreModelGrader = object { input, model, name, 3 more }` + - `ScoreModelGrader object { input, model, name, 3 more }` A ScoreModelGrader object that uses a model to assign a score to the input. @@ -1965,7 +1965,7 @@ Response includes details of the enqueued job including job status and the name A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. @@ -1979,7 +1979,7 @@ Response includes details of the enqueued job including job status and the name - `"input_text"` - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -1993,7 +1993,7 @@ Response includes details of the enqueued job including job status and the name - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -2011,7 +2011,7 @@ Response includes details of the enqueued job including job status and the name The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. @@ -2045,21 +2045,11 @@ Response includes details of the enqueued job including job status and the name A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -2073,7 +2063,7 @@ Response includes details of the enqueued job including job status and the name - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -2091,31 +2081,10 @@ Response includes details of the enqueued job including job status and the name The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or @@ -2198,116 +2167,54 @@ Response includes details of the enqueued job including job status and the name An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `LabelModelGrader = object { input, labels, model, 3 more }` - - A LabelModelGrader object which uses a model to assign labels to each item - in the evaluation. - - - `input: array of object { content, role, type }` - - - `content: string or ResponseInputText or object { text, type } or 3 more` - - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` + - `MultiGrader object { calculate_output, graders, name, type }` - The type of the image input. Always `input_image`. + A MultiGrader object combines the output of multiple graders to produce a single score. - - `"input_image"` + - `calculate_output: string` - - `detail: optional string` + A formula to calculate the output based on grader results. - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + - `graders: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - - `ResponseInputAudio = object { input_audio, type }` + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - An audio input to the model. + - `StringCheckGrader object { input, name, operation, 2 more }` - - `input_audio: object { data, format }` + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - `data: string` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` - Base64-encoded audio data. + A TextSimilarityGrader object which grades text based on similarity metrics. - - `format: "mp3" or "wav"` + - `PythonGrader object { name, source, type, image_tag }` - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + A PythonGrader object that runs a python script on the input. - - `"mp3"` + - `ScoreModelGrader object { input, model, name, 3 more }` - - `"wav"` + A ScoreModelGrader object that uses a model to assign a score to the input. - - `type: "input_audio"` + - `LabelModelGrader object { input, labels, model, 3 more }` - The type of the input item. Always `input_audio`. + A LabelModelGrader object which uses a model to assign labels to each item + in the evaluation. - - `"input_audio"` + - `input: array of object { content, role, type }` - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` + - `content: string or ResponseInputText or object { text, type } or 3 more` - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. + Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - `TextInput = string` A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -2321,7 +2228,7 @@ Response includes details of the enqueued job including job status and the name - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -2339,30 +2246,14 @@ Response includes details of the enqueued job including job status and the name The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. + - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - `"input_audio"` + A list of inputs, each of which may be either an input text, output text, input + image, or input audio object. - `role: "user" or "assistant" or "system" or "developer"` @@ -2529,6207 +2420,416 @@ Response includes details of the enqueued job including job status and the name - `number` -- `seed: optional number` - - The seed controls the reproducibility of the job. Passing in the same seed and job parameters should produce the same results, but may differ in rare cases. - If a seed is not specified, one will be generated for you. - -- `suffix: optional string` - - A string of up to 64 characters that will be added to your fine-tuned model name. - - For example, a `suffix` of "custom-model-name" would produce a model name like `ft:gpt-4o-mini:openai:custom-model-name:7p4lURel`. - -- `validation_file: optional string` - - The ID of an uploaded file that contains validation data. - - If you provide this file, the data is used to generate validation - metrics periodically during fine-tuning. These metrics can be viewed in - the fine-tuning results file. - The same data should not be present in both train and validation files. - - Your dataset must be formatted as a JSONL file. You must upload your file with the purpose `fine-tune`. - - See the [fine-tuning guide](/docs/guides/model-optimization) for more details. - -### Returns - -- `FineTuningJob = object { id, created_at, error, 16 more }` - - The `fine_tuning.job` object represents a fine-tuning job that has been created through the API. - - - `id: string` - - The object identifier, which can be referenced in the API endpoints. - - - `created_at: number` - - The Unix timestamp (in seconds) for when the fine-tuning job was created. - - - `error: object { code, message, param }` - - For fine-tuning jobs that have `failed`, this will contain more information on the cause of the failure. - - - `code: string` - - A machine-readable error code. - - - `message: string` - - A human-readable error message. - - - `param: string` - - The parameter that was invalid, usually `training_file` or `validation_file`. This field will be null if the failure was not parameter-specific. - - - `fine_tuned_model: string` - - The name of the fine-tuned model that is being created. The value will be null if the fine-tuning job is still running. - - - `finished_at: number` - - The Unix timestamp (in seconds) for when the fine-tuning job was finished. The value will be null if the fine-tuning job is still running. - - - `hyperparameters: object { batch_size, learning_rate_multiplier, n_epochs }` +### Example - The hyperparameters used for the fine-tuning job. This value will only be returned when running `supervised` jobs. +```http +curl https://api.openai.com/v1/fine_tuning/jobs \ + -H 'Content-Type: application/json' \ + -H "Authorization: Bearer $OPENAI_API_KEY" \ + -d '{ + "model": "gpt-4o-mini", + "training_file": "file-abc123", + "seed": 42, + "validation_file": "file-abc123" + }' +``` - - `batch_size: optional "auto" or number` +#### Response - Number of examples in each batch. A larger batch size means that model parameters - are updated less frequently, but with lower variance. - - - `"auto"` - - - `"auto"` - - - `number` - - - `learning_rate_multiplier: optional "auto" or number` - - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid - overfitting. - - - `"auto"` - - - `"auto"` - - - `number` - - - `n_epochs: optional "auto" or number` - - The number of epochs to train the model for. An epoch refers to one full cycle - through the training dataset. - - - `"auto"` - - - `"auto"` - - - `number` - - - `model: string` - - The base model that is being fine-tuned. - - - `object: "fine_tuning.job"` - - The object type, which is always "fine_tuning.job". - - - `"fine_tuning.job"` - - - `organization_id: string` - - The organization that owns the fine-tuning job. - - - `result_files: array of string` - - The compiled results file ID(s) for the fine-tuning job. You can retrieve the results with the [Files API](/docs/api-reference/files/retrieve-contents). - - - `seed: number` - - The seed used for the fine-tuning job. - - - `status: "validating_files" or "queued" or "running" or 3 more` - - The current status of the fine-tuning job, which can be either `validating_files`, `queued`, `running`, `succeeded`, `failed`, or `cancelled`. - - - `"validating_files"` - - - `"queued"` - - - `"running"` - - - `"succeeded"` - - - `"failed"` - - - `"cancelled"` - - - `trained_tokens: number` - - The total number of billable tokens processed by this fine-tuning job. The value will be null if the fine-tuning job is still running. - - - `training_file: string` - - The file ID used for training. You can retrieve the training data with the [Files API](/docs/api-reference/files/retrieve-contents). - - - `validation_file: string` - - The file ID used for validation. You can retrieve the validation results with the [Files API](/docs/api-reference/files/retrieve-contents). - - - `estimated_finish: optional number` - - The Unix timestamp (in seconds) for when the fine-tuning job is estimated to finish. The value will be null if the fine-tuning job is not running. - - - `integrations: optional array of FineTuningJobWandbIntegrationObject` - - A list of integrations to enable for this fine-tuning job. - - - `type: "wandb"` - - The type of the integration being enabled for the fine-tuning job - - - `"wandb"` - - - `wandb: FineTuningJobWandbIntegration` - - The settings for your integration with Weights and Biases. This payload specifies the project that - metrics will be sent to. Optionally, you can set an explicit display name for your run, add tags - to your run, and set a default entity (team, username, etc) to be associated with your run. - - - `project: string` - - The name of the project that the new run will be created under. - - - `entity: optional string` - - The entity to use for the run. This allows you to set the team or username of the WandB user that you would - like associated with the run. If not set, the default entity for the registered WandB API key is used. - - - `name: optional string` - - A display name to set for the run. If not set, we will use the Job ID as the name. - - - `tags: optional array of string` - - A list of tags to be attached to the newly created run. These tags are passed through directly to WandB. Some - default tags are generated by OpenAI: "openai/finetune", "openai/{base-model}", "openai/{ftjob-abcdef}". - - - `metadata: optional Metadata` - - Set of 16 key-value pairs that can be attached to an object. This can be - useful for storing additional information about the object in a structured - format, and querying for objects via API or the dashboard. - - Keys are strings with a maximum length of 64 characters. Values are strings - with a maximum length of 512 characters. - - - `method: optional object { type, dpo, reinforcement, supervised }` - - The method used for fine-tuning. - - - `type: "supervised" or "dpo" or "reinforcement"` - - The type of method. Is either `supervised`, `dpo`, or `reinforcement`. - - - `"supervised"` - - - `"dpo"` - - - `"reinforcement"` - - - `dpo: optional DpoMethod` - - Configuration for the DPO fine-tuning method. - - - `hyperparameters: optional DpoHyperparameters` - - The hyperparameters used for the DPO fine-tuning job. - - - `batch_size: optional "auto" or number` - - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - - `"auto"` - - - `"auto"` - - - `number` - - - `beta: optional "auto" or number` - - The beta value for the DPO method. A higher beta value will increase the weight of the penalty between the policy and reference model. - - - `"auto"` - - - `"auto"` - - - `number` - - - `learning_rate_multiplier: optional "auto" or number` - - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - - `"auto"` - - - `"auto"` - - - `number` - - - `n_epochs: optional "auto" or number` - - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - - `"auto"` - - - `"auto"` - - - `number` - - - `reinforcement: optional ReinforcementMethod` - - Configuration for the reinforcement fine-tuning method. - - - `grader: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - - The grader used for the fine-tuning job. - - - `StringCheckGrader = object { input, name, operation, 2 more }` - - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - - `input: string` - - The input text. This may include template strings. - - - `name: string` - - The name of the grader. - - - `operation: "eq" or "ne" or "like" or "ilike"` - - The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. - - - `"eq"` - - - `"ne"` - - - `"like"` - - - `"ilike"` - - - `reference: string` - - The reference text. This may include template strings. - - - `type: "string_check"` - - The object type, which is always `string_check`. - - - `"string_check"` - - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` - - A TextSimilarityGrader object which grades text based on similarity metrics. - - - `evaluation_metric: "cosine" or "fuzzy_match" or "bleu" or 8 more` - - The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, - `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, - or `rouge_l`. - - - `"cosine"` - - - `"fuzzy_match"` - - - `"bleu"` - - - `"gleu"` - - - `"meteor"` - - - `"rouge_1"` - - - `"rouge_2"` - - - `"rouge_3"` - - - `"rouge_4"` - - - `"rouge_5"` - - - `"rouge_l"` - - - `input: string` - - The text being graded. - - - `name: string` - - The name of the grader. - - - `reference: string` - - The text being graded against. - - - `type: "text_similarity"` - - The type of grader. - - - `"text_similarity"` - - - `PythonGrader = object { name, source, type, image_tag }` - - A PythonGrader object that runs a python script on the input. - - - `name: string` - - The name of the grader. - - - `source: string` - - The source code of the python script. - - - `type: "python"` - - The object type, which is always `python`. - - - `"python"` - - - `image_tag: optional string` - - The image tag to use for the python script. - - - `ScoreModelGrader = object { input, model, name, 3 more }` - - A ScoreModelGrader object that uses a model to assign a score to the input. - - - `input: array of object { content, role, type }` - - The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. - - - `content: string or ResponseInputText or object { text, type } or 3 more` - - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `role: "user" or "assistant" or "system" or "developer"` - - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. - - - `"user"` - - - `"assistant"` - - - `"system"` - - - `"developer"` - - - `type: optional "message"` - - The type of the message input. Always `message`. - - - `"message"` - - - `model: string` - - The model to use for the evaluation. - - - `name: string` - - The name of the grader. - - - `type: "score_model"` - - The object type, which is always `score_model`. - - - `"score_model"` - - - `range: optional array of number` - - The range of the score. Defaults to `[0, 1]`. - - - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` - - The sampling parameters for the model. - - - `max_completions_tokens: optional number` - - The maximum number of tokens the grader model may generate in its response. - - - `reasoning_effort: optional ReasoningEffort` - - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. - - - `"none"` - - - `"minimal"` - - - `"low"` - - - `"medium"` - - - `"high"` - - - `"xhigh"` - - - `seed: optional number` - - A seed value to initialize the randomness, during sampling. - - - `temperature: optional number` - - A higher temperature increases randomness in the outputs. - - - `top_p: optional number` - - An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - - `MultiGrader = object { calculate_output, graders, name, type }` - - A MultiGrader object combines the output of multiple graders to produce a single score. - - - `calculate_output: string` - - A formula to calculate the output based on grader results. - - - `graders: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - - `StringCheckGrader = object { input, name, operation, 2 more }` - - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - - `input: string` - - The input text. This may include template strings. - - - `name: string` - - The name of the grader. - - - `operation: "eq" or "ne" or "like" or "ilike"` - - The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. - - - `"eq"` - - - `"ne"` - - - `"like"` - - - `"ilike"` - - - `reference: string` - - The reference text. This may include template strings. - - - `type: "string_check"` - - The object type, which is always `string_check`. - - - `"string_check"` - - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` - - A TextSimilarityGrader object which grades text based on similarity metrics. - - - `evaluation_metric: "cosine" or "fuzzy_match" or "bleu" or 8 more` - - The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, - `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, - or `rouge_l`. - - - `"cosine"` - - - `"fuzzy_match"` - - - `"bleu"` - - - `"gleu"` - - - `"meteor"` - - - `"rouge_1"` - - - `"rouge_2"` - - - `"rouge_3"` - - - `"rouge_4"` - - - `"rouge_5"` - - - `"rouge_l"` - - - `input: string` - - The text being graded. - - - `name: string` - - The name of the grader. - - - `reference: string` - - The text being graded against. - - - `type: "text_similarity"` - - The type of grader. - - - `"text_similarity"` - - - `PythonGrader = object { name, source, type, image_tag }` - - A PythonGrader object that runs a python script on the input. - - - `name: string` - - The name of the grader. - - - `source: string` - - The source code of the python script. - - - `type: "python"` - - The object type, which is always `python`. - - - `"python"` - - - `image_tag: optional string` - - The image tag to use for the python script. - - - `ScoreModelGrader = object { input, model, name, 3 more }` - - A ScoreModelGrader object that uses a model to assign a score to the input. - - - `input: array of object { content, role, type }` - - The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. - - - `content: string or ResponseInputText or object { text, type } or 3 more` - - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `role: "user" or "assistant" or "system" or "developer"` - - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. - - - `"user"` - - - `"assistant"` - - - `"system"` - - - `"developer"` - - - `type: optional "message"` - - The type of the message input. Always `message`. - - - `"message"` - - - `model: string` - - The model to use for the evaluation. - - - `name: string` - - The name of the grader. - - - `type: "score_model"` - - The object type, which is always `score_model`. - - - `"score_model"` - - - `range: optional array of number` - - The range of the score. Defaults to `[0, 1]`. - - - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` - - The sampling parameters for the model. - - - `max_completions_tokens: optional number` - - The maximum number of tokens the grader model may generate in its response. - - - `reasoning_effort: optional ReasoningEffort` - - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. - - - `"none"` - - - `"minimal"` - - - `"low"` - - - `"medium"` - - - `"high"` - - - `"xhigh"` - - - `seed: optional number` - - A seed value to initialize the randomness, during sampling. - - - `temperature: optional number` - - A higher temperature increases randomness in the outputs. - - - `top_p: optional number` - - An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - - `LabelModelGrader = object { input, labels, model, 3 more }` - - A LabelModelGrader object which uses a model to assign labels to each item - in the evaluation. - - - `input: array of object { content, role, type }` - - - `content: string or ResponseInputText or object { text, type } or 3 more` - - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `role: "user" or "assistant" or "system" or "developer"` - - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. - - - `"user"` - - - `"assistant"` - - - `"system"` - - - `"developer"` - - - `type: optional "message"` - - The type of the message input. Always `message`. - - - `"message"` - - - `labels: array of string` - - The labels to assign to each item in the evaluation. - - - `model: string` - - The model to use for the evaluation. Must support structured outputs. - - - `name: string` - - The name of the grader. - - - `passing_labels: array of string` - - The labels that indicate a passing result. Must be a subset of labels. - - - `type: "label_model"` - - The object type, which is always `label_model`. - - - `"label_model"` - - - `name: string` - - The name of the grader. - - - `type: "multi"` - - The object type, which is always `multi`. - - - `"multi"` - - - `hyperparameters: optional ReinforcementHyperparameters` - - The hyperparameters used for the reinforcement fine-tuning job. - - - `batch_size: optional "auto" or number` - - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - - `"auto"` - - - `"auto"` - - - `number` - - - `compute_multiplier: optional "auto" or number` - - Multiplier on amount of compute used for exploring search space during training. - - - `"auto"` - - - `"auto"` - - - `number` - - - `eval_interval: optional "auto" or number` - - The number of training steps between evaluation runs. - - - `"auto"` - - - `"auto"` - - - `number` - - - `eval_samples: optional "auto" or number` - - Number of evaluation samples to generate per training step. - - - `"auto"` - - - `"auto"` - - - `number` - - - `learning_rate_multiplier: optional "auto" or number` - - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - - `"auto"` - - - `"auto"` - - - `number` - - - `n_epochs: optional "auto" or number` - - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - - `"auto"` - - - `"auto"` - - - `number` - - - `reasoning_effort: optional "default" or "low" or "medium" or "high"` - - Level of reasoning effort. - - - `"default"` - - - `"low"` - - - `"medium"` - - - `"high"` - - - `supervised: optional SupervisedMethod` - - Configuration for the supervised fine-tuning method. - - - `hyperparameters: optional SupervisedHyperparameters` - - The hyperparameters used for the fine-tuning job. - - - `batch_size: optional "auto" or number` - - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - - `"auto"` - - - `"auto"` - - - `number` - - - `learning_rate_multiplier: optional "auto" or number` - - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - - `"auto"` - - - `"auto"` - - - `number` - - - `n_epochs: optional "auto" or number` - - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - - `"auto"` - - - `"auto"` - - - `number` - -### Example - -```http -curl https://api.openai.com/v1/fine_tuning/jobs \ - -H 'Content-Type: application/json' \ - -H "Authorization: Bearer $OPENAI_API_KEY" \ - -d '{ - "model": "gpt-4o-mini", - "training_file": "file-abc123", - "seed": 42, - "validation_file": "file-abc123" - }' -``` - -#### Response - -```json -{ - "id": "id", - "created_at": 0, - "error": { - "code": "code", - "message": "message", - "param": "param" - }, - "fine_tuned_model": "fine_tuned_model", - "finished_at": 0, - "hyperparameters": { - "batch_size": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto" - }, - "model": "model", - "object": "fine_tuning.job", - "organization_id": "organization_id", - "result_files": [ - "file-abc123" - ], - "seed": 0, - "status": "validating_files", - "trained_tokens": 0, - "training_file": "training_file", - "validation_file": "validation_file", - "estimated_finish": 0, - "integrations": [ - { - "type": "wandb", - "wandb": { - "project": "my-wandb-project", - "entity": "entity", - "name": "name", - "tags": [ - "custom-tag" - ] - } - } - ], - "metadata": { - "foo": "string" - }, - "method": { - "type": "supervised", - "dpo": { - "hyperparameters": { - "batch_size": "auto", - "beta": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto" - } - }, - "reinforcement": { - "grader": { - "input": "input", - "name": "name", - "operation": "eq", - "reference": "reference", - "type": "string_check" - }, - "hyperparameters": { - "batch_size": "auto", - "compute_multiplier": "auto", - "eval_interval": "auto", - "eval_samples": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto", - "reasoning_effort": "default" - } - }, - "supervised": { - "hyperparameters": { - "batch_size": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto" - } - } - } -} -``` - -### Example - -```http -curl https://api.openai.com/v1/fine_tuning/jobs \ - -H "Content-Type: application/json" \ - -H "Authorization: Bearer $OPENAI_API_KEY" \ - -d '{ - "training_file": "file-BK7bzQj3FfZFXr7DbL6xJwfo", - "model": "gpt-4o-mini" - }' -``` - -#### Response - -```json -{ - "object": "fine_tuning.job", - "id": "ftjob-abc123", - "model": "gpt-4o-mini-2024-07-18", - "created_at": 1721764800, - "fine_tuned_model": null, - "organization_id": "org-123", - "result_files": [], - "status": "queued", - "validation_file": null, - "training_file": "file-abc123", - "method": { - "type": "supervised", - "supervised": { - "hyperparameters": { - "batch_size": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto", - } - } - }, - "metadata": null -} -``` - -### Epochs - -```http -curl https://api.openai.com/v1/fine_tuning/jobs \ - -H "Content-Type: application/json" \ - -H "Authorization: Bearer $OPENAI_API_KEY" \ - -d '{ - "training_file": "file-abc123", - "model": "gpt-4o-mini", - "method": { - "type": "supervised", - "supervised": { - "hyperparameters": { - "n_epochs": 2 - } - } - } - }' -``` - -#### Response - -```json -{ - "object": "fine_tuning.job", - "id": "ftjob-abc123", - "model": "gpt-4o-mini", - "created_at": 1721764800, - "fine_tuned_model": null, - "organization_id": "org-123", - "result_files": [], - "status": "queued", - "validation_file": null, - "training_file": "file-abc123", - "hyperparameters": { - "batch_size": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": 2 - }, - "method": { - "type": "supervised", - "supervised": { - "hyperparameters": { - "batch_size": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": 2 - } - } - }, - "metadata": null, - "error": { - "code": null, - "message": null, - "param": null - }, - "finished_at": null, - "seed": 683058546, - "trained_tokens": null, - "estimated_finish": null, - "integrations": [], - "user_provided_suffix": null, - "usage_metrics": null, - "shared_with_openai": false -} -``` - -### DPO - -```http -curl https://api.openai.com/v1/fine_tuning/jobs \ - -H "Content-Type: application/json" \ - -H "Authorization: Bearer $OPENAI_API_KEY" \ - -d '{ - "training_file": "file-abc123", - "validation_file": "file-abc123", - "model": "gpt-4o-mini", - "method": { - "type": "dpo", - "dpo": { - "hyperparameters": { - "beta": 0.1 - } - } - } - }' -``` - -#### Response - -```json -{ - "object": "fine_tuning.job", - "id": "ftjob-abc", - "model": "gpt-4o-mini", - "created_at": 1746130590, - "fine_tuned_model": null, - "organization_id": "org-abc", - "result_files": [], - "status": "queued", - "validation_file": "file-123", - "training_file": "file-abc", - "method": { - "type": "dpo", - "dpo": { - "hyperparameters": { - "beta": 0.1, - "batch_size": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto" - } - } - }, - "metadata": null, - "error": { - "code": null, - "message": null, - "param": null - }, - "finished_at": null, - "hyperparameters": null, - "seed": 1036326793, - "estimated_finish": null, - "integrations": [], - "user_provided_suffix": null, - "usage_metrics": null, - "shared_with_openai": false -} -``` - -### Reinforcement - -```http -curl https://api.openai.com/v1/fine_tuning/jobs \ - -H "Content-Type: application/json" \ - -H "Authorization: Bearer $OPENAI_API_KEY" \ - -d '{ - "training_file": "file-abc", - "validation_file": "file-123", - "model": "o4-mini", - "method": { - "type": "reinforcement", - "reinforcement": { - "grader": { - "type": "string_check", - "name": "Example string check grader", - "input": "{{sample.output_text}}", - "reference": "{{item.label}}", - "operation": "eq" - }, - "hyperparameters": { - "reasoning_effort": "medium" - } - } - } - }' -``` - -#### Response - -```json -{ - "object": "fine_tuning.job", - "id": "ftjob-abc123", - "model": "o4-mini", - "created_at": 1721764800, - "finished_at": null, - "fine_tuned_model": null, - "organization_id": "org-123", - "result_files": [], - "status": "validating_files", - "validation_file": "file-123", - "training_file": "file-abc", - "trained_tokens": null, - "error": {}, - "user_provided_suffix": null, - "seed": 950189191, - "estimated_finish": null, - "integrations": [], - "method": { - "type": "reinforcement", - "reinforcement": { - "hyperparameters": { - "batch_size": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto", - "eval_interval": "auto", - "eval_samples": "auto", - "compute_multiplier": "auto", - "reasoning_effort": "medium" - }, - "grader": { - "type": "string_check", - "name": "Example string check grader", - "input": "{{sample.output_text}}", - "reference": "{{item.label}}", - "operation": "eq" - }, - "response_format": null - } - }, - "metadata": null, - "usage_metrics": null, - "shared_with_openai": false -} - -``` - -### Validation file - -```http -curl https://api.openai.com/v1/fine_tuning/jobs \ - -H "Content-Type: application/json" \ - -H "Authorization: Bearer $OPENAI_API_KEY" \ - -d '{ - "training_file": "file-abc123", - "validation_file": "file-abc123", - "model": "gpt-4o-mini" - }' -``` - -#### Response - -```json -{ - "object": "fine_tuning.job", - "id": "ftjob-abc123", - "model": "gpt-4o-mini-2024-07-18", - "created_at": 1721764800, - "fine_tuned_model": null, - "organization_id": "org-123", - "result_files": [], - "status": "queued", - "validation_file": "file-abc123", - "training_file": "file-abc123", - "method": { - "type": "supervised", - "supervised": { - "hyperparameters": { - "batch_size": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto", - } - } - }, - "metadata": null -} -``` - -### W&B Integration - -```http -curl https://api.openai.com/v1/fine_tuning/jobs \ - -H "Content-Type: application/json" \ - -H "Authorization: Bearer $OPENAI_API_KEY" \ - -d '{ - "training_file": "file-abc123", - "validation_file": "file-abc123", - "model": "gpt-4o-mini", - "integrations": [ - { - "type": "wandb", - "wandb": { - "project": "my-wandb-project", - "name": "ft-run-display-name" - "tags": [ - "first-experiment", "v2" - ] - } - } - ] - }' -``` - -#### Response - -```json -{ - "object": "fine_tuning.job", - "id": "ftjob-abc123", - "model": "gpt-4o-mini-2024-07-18", - "created_at": 1721764800, - "fine_tuned_model": null, - "organization_id": "org-123", - "result_files": [], - "status": "queued", - "validation_file": "file-abc123", - "training_file": "file-abc123", - "integrations": [ - { - "type": "wandb", - "wandb": { - "project": "my-wandb-project", - "entity": None, - "run_id": "ftjob-abc123" - } - } - ], - "method": { - "type": "supervised", - "supervised": { - "hyperparameters": { - "batch_size": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto", - } - } - }, - "metadata": null -} -``` - -## List fine-tuning jobs - -**get** `/fine_tuning/jobs` - -List your organization's fine-tuning jobs - -### Query Parameters - -- `after: optional string` - - Identifier for the last job from the previous pagination request. - -- `limit: optional number` - - Number of fine-tuning jobs to retrieve. - -- `metadata: optional map[string]` - - Optional metadata filter. To filter, use the syntax `metadata[k]=v`. Alternatively, set `metadata=null` to indicate no metadata. - -### Returns - -- `data: array of FineTuningJob` - - - `id: string` - - The object identifier, which can be referenced in the API endpoints. - - - `created_at: number` - - The Unix timestamp (in seconds) for when the fine-tuning job was created. - - - `error: object { code, message, param }` - - For fine-tuning jobs that have `failed`, this will contain more information on the cause of the failure. - - - `code: string` - - A machine-readable error code. - - - `message: string` - - A human-readable error message. - - - `param: string` - - The parameter that was invalid, usually `training_file` or `validation_file`. This field will be null if the failure was not parameter-specific. - - - `fine_tuned_model: string` - - The name of the fine-tuned model that is being created. The value will be null if the fine-tuning job is still running. - - - `finished_at: number` - - The Unix timestamp (in seconds) for when the fine-tuning job was finished. The value will be null if the fine-tuning job is still running. - - - `hyperparameters: object { batch_size, learning_rate_multiplier, n_epochs }` - - The hyperparameters used for the fine-tuning job. This value will only be returned when running `supervised` jobs. - - - `batch_size: optional "auto" or number` - - Number of examples in each batch. A larger batch size means that model parameters - are updated less frequently, but with lower variance. - - - `"auto"` - - - `"auto"` - - - `number` - - - `learning_rate_multiplier: optional "auto" or number` - - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid - overfitting. - - - `"auto"` - - - `"auto"` - - - `number` - - - `n_epochs: optional "auto" or number` - - The number of epochs to train the model for. An epoch refers to one full cycle - through the training dataset. - - - `"auto"` - - - `"auto"` - - - `number` - - - `model: string` - - The base model that is being fine-tuned. - - - `object: "fine_tuning.job"` - - The object type, which is always "fine_tuning.job". - - - `"fine_tuning.job"` - - - `organization_id: string` - - The organization that owns the fine-tuning job. - - - `result_files: array of string` - - The compiled results file ID(s) for the fine-tuning job. You can retrieve the results with the [Files API](/docs/api-reference/files/retrieve-contents). - - - `seed: number` - - The seed used for the fine-tuning job. - - - `status: "validating_files" or "queued" or "running" or 3 more` - - The current status of the fine-tuning job, which can be either `validating_files`, `queued`, `running`, `succeeded`, `failed`, or `cancelled`. - - - `"validating_files"` - - - `"queued"` - - - `"running"` - - - `"succeeded"` - - - `"failed"` - - - `"cancelled"` - - - `trained_tokens: number` - - The total number of billable tokens processed by this fine-tuning job. The value will be null if the fine-tuning job is still running. - - - `training_file: string` - - The file ID used for training. You can retrieve the training data with the [Files API](/docs/api-reference/files/retrieve-contents). - - - `validation_file: string` - - The file ID used for validation. You can retrieve the validation results with the [Files API](/docs/api-reference/files/retrieve-contents). - - - `estimated_finish: optional number` - - The Unix timestamp (in seconds) for when the fine-tuning job is estimated to finish. The value will be null if the fine-tuning job is not running. - - - `integrations: optional array of FineTuningJobWandbIntegrationObject` - - A list of integrations to enable for this fine-tuning job. - - - `type: "wandb"` - - The type of the integration being enabled for the fine-tuning job - - - `"wandb"` - - - `wandb: FineTuningJobWandbIntegration` - - The settings for your integration with Weights and Biases. This payload specifies the project that - metrics will be sent to. Optionally, you can set an explicit display name for your run, add tags - to your run, and set a default entity (team, username, etc) to be associated with your run. - - - `project: string` - - The name of the project that the new run will be created under. - - - `entity: optional string` - - The entity to use for the run. This allows you to set the team or username of the WandB user that you would - like associated with the run. If not set, the default entity for the registered WandB API key is used. - - - `name: optional string` - - A display name to set for the run. If not set, we will use the Job ID as the name. - - - `tags: optional array of string` - - A list of tags to be attached to the newly created run. These tags are passed through directly to WandB. Some - default tags are generated by OpenAI: "openai/finetune", "openai/{base-model}", "openai/{ftjob-abcdef}". - - - `metadata: optional Metadata` - - Set of 16 key-value pairs that can be attached to an object. This can be - useful for storing additional information about the object in a structured - format, and querying for objects via API or the dashboard. - - Keys are strings with a maximum length of 64 characters. Values are strings - with a maximum length of 512 characters. - - - `method: optional object { type, dpo, reinforcement, supervised }` - - The method used for fine-tuning. - - - `type: "supervised" or "dpo" or "reinforcement"` - - The type of method. Is either `supervised`, `dpo`, or `reinforcement`. - - - `"supervised"` - - - `"dpo"` - - - `"reinforcement"` - - - `dpo: optional DpoMethod` - - Configuration for the DPO fine-tuning method. - - - `hyperparameters: optional DpoHyperparameters` - - The hyperparameters used for the DPO fine-tuning job. - - - `batch_size: optional "auto" or number` - - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - - `"auto"` - - - `"auto"` - - - `number` - - - `beta: optional "auto" or number` - - The beta value for the DPO method. A higher beta value will increase the weight of the penalty between the policy and reference model. - - - `"auto"` - - - `"auto"` - - - `number` - - - `learning_rate_multiplier: optional "auto" or number` - - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - - `"auto"` - - - `"auto"` - - - `number` - - - `n_epochs: optional "auto" or number` - - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - - `"auto"` - - - `"auto"` - - - `number` - - - `reinforcement: optional ReinforcementMethod` - - Configuration for the reinforcement fine-tuning method. - - - `grader: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - - The grader used for the fine-tuning job. - - - `StringCheckGrader = object { input, name, operation, 2 more }` - - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - - `input: string` - - The input text. This may include template strings. - - - `name: string` - - The name of the grader. - - - `operation: "eq" or "ne" or "like" or "ilike"` - - The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. - - - `"eq"` - - - `"ne"` - - - `"like"` - - - `"ilike"` - - - `reference: string` - - The reference text. This may include template strings. - - - `type: "string_check"` - - The object type, which is always `string_check`. - - - `"string_check"` - - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` - - A TextSimilarityGrader object which grades text based on similarity metrics. - - - `evaluation_metric: "cosine" or "fuzzy_match" or "bleu" or 8 more` - - The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, - `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, - or `rouge_l`. - - - `"cosine"` - - - `"fuzzy_match"` - - - `"bleu"` - - - `"gleu"` - - - `"meteor"` - - - `"rouge_1"` - - - `"rouge_2"` - - - `"rouge_3"` - - - `"rouge_4"` - - - `"rouge_5"` - - - `"rouge_l"` - - - `input: string` - - The text being graded. - - - `name: string` - - The name of the grader. - - - `reference: string` - - The text being graded against. - - - `type: "text_similarity"` - - The type of grader. - - - `"text_similarity"` - - - `PythonGrader = object { name, source, type, image_tag }` - - A PythonGrader object that runs a python script on the input. - - - `name: string` - - The name of the grader. - - - `source: string` - - The source code of the python script. - - - `type: "python"` - - The object type, which is always `python`. - - - `"python"` - - - `image_tag: optional string` - - The image tag to use for the python script. - - - `ScoreModelGrader = object { input, model, name, 3 more }` - - A ScoreModelGrader object that uses a model to assign a score to the input. - - - `input: array of object { content, role, type }` - - The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. - - - `content: string or ResponseInputText or object { text, type } or 3 more` - - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `role: "user" or "assistant" or "system" or "developer"` - - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. - - - `"user"` - - - `"assistant"` - - - `"system"` - - - `"developer"` - - - `type: optional "message"` - - The type of the message input. Always `message`. - - - `"message"` - - - `model: string` - - The model to use for the evaluation. - - - `name: string` - - The name of the grader. - - - `type: "score_model"` - - The object type, which is always `score_model`. - - - `"score_model"` - - - `range: optional array of number` - - The range of the score. Defaults to `[0, 1]`. - - - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` - - The sampling parameters for the model. - - - `max_completions_tokens: optional number` - - The maximum number of tokens the grader model may generate in its response. - - - `reasoning_effort: optional ReasoningEffort` - - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. - - - `"none"` - - - `"minimal"` - - - `"low"` - - - `"medium"` - - - `"high"` - - - `"xhigh"` - - - `seed: optional number` - - A seed value to initialize the randomness, during sampling. - - - `temperature: optional number` - - A higher temperature increases randomness in the outputs. - - - `top_p: optional number` - - An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - - `MultiGrader = object { calculate_output, graders, name, type }` - - A MultiGrader object combines the output of multiple graders to produce a single score. - - - `calculate_output: string` - - A formula to calculate the output based on grader results. - - - `graders: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - - `StringCheckGrader = object { input, name, operation, 2 more }` - - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - - `input: string` - - The input text. This may include template strings. - - - `name: string` - - The name of the grader. - - - `operation: "eq" or "ne" or "like" or "ilike"` - - The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. - - - `"eq"` - - - `"ne"` - - - `"like"` - - - `"ilike"` - - - `reference: string` - - The reference text. This may include template strings. - - - `type: "string_check"` - - The object type, which is always `string_check`. - - - `"string_check"` - - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` - - A TextSimilarityGrader object which grades text based on similarity metrics. - - - `evaluation_metric: "cosine" or "fuzzy_match" or "bleu" or 8 more` - - The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, - `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, - or `rouge_l`. - - - `"cosine"` - - - `"fuzzy_match"` - - - `"bleu"` - - - `"gleu"` - - - `"meteor"` - - - `"rouge_1"` - - - `"rouge_2"` - - - `"rouge_3"` - - - `"rouge_4"` - - - `"rouge_5"` - - - `"rouge_l"` - - - `input: string` - - The text being graded. - - - `name: string` - - The name of the grader. - - - `reference: string` - - The text being graded against. - - - `type: "text_similarity"` - - The type of grader. - - - `"text_similarity"` - - - `PythonGrader = object { name, source, type, image_tag }` - - A PythonGrader object that runs a python script on the input. - - - `name: string` - - The name of the grader. - - - `source: string` - - The source code of the python script. - - - `type: "python"` - - The object type, which is always `python`. - - - `"python"` - - - `image_tag: optional string` - - The image tag to use for the python script. - - - `ScoreModelGrader = object { input, model, name, 3 more }` - - A ScoreModelGrader object that uses a model to assign a score to the input. - - - `input: array of object { content, role, type }` - - The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. - - - `content: string or ResponseInputText or object { text, type } or 3 more` - - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `role: "user" or "assistant" or "system" or "developer"` - - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. - - - `"user"` - - - `"assistant"` - - - `"system"` - - - `"developer"` - - - `type: optional "message"` - - The type of the message input. Always `message`. - - - `"message"` - - - `model: string` - - The model to use for the evaluation. - - - `name: string` - - The name of the grader. - - - `type: "score_model"` - - The object type, which is always `score_model`. - - - `"score_model"` - - - `range: optional array of number` - - The range of the score. Defaults to `[0, 1]`. - - - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` - - The sampling parameters for the model. - - - `max_completions_tokens: optional number` - - The maximum number of tokens the grader model may generate in its response. - - - `reasoning_effort: optional ReasoningEffort` - - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. - - - `"none"` - - - `"minimal"` - - - `"low"` - - - `"medium"` - - - `"high"` - - - `"xhigh"` - - - `seed: optional number` - - A seed value to initialize the randomness, during sampling. - - - `temperature: optional number` - - A higher temperature increases randomness in the outputs. - - - `top_p: optional number` - - An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - - `LabelModelGrader = object { input, labels, model, 3 more }` - - A LabelModelGrader object which uses a model to assign labels to each item - in the evaluation. - - - `input: array of object { content, role, type }` - - - `content: string or ResponseInputText or object { text, type } or 3 more` - - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `role: "user" or "assistant" or "system" or "developer"` - - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. - - - `"user"` - - - `"assistant"` - - - `"system"` - - - `"developer"` - - - `type: optional "message"` - - The type of the message input. Always `message`. - - - `"message"` - - - `labels: array of string` - - The labels to assign to each item in the evaluation. - - - `model: string` - - The model to use for the evaluation. Must support structured outputs. - - - `name: string` - - The name of the grader. - - - `passing_labels: array of string` - - The labels that indicate a passing result. Must be a subset of labels. - - - `type: "label_model"` - - The object type, which is always `label_model`. - - - `"label_model"` - - - `name: string` - - The name of the grader. - - - `type: "multi"` - - The object type, which is always `multi`. - - - `"multi"` - - - `hyperparameters: optional ReinforcementHyperparameters` - - The hyperparameters used for the reinforcement fine-tuning job. - - - `batch_size: optional "auto" or number` - - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - - `"auto"` - - - `"auto"` - - - `number` - - - `compute_multiplier: optional "auto" or number` - - Multiplier on amount of compute used for exploring search space during training. - - - `"auto"` - - - `"auto"` - - - `number` - - - `eval_interval: optional "auto" or number` - - The number of training steps between evaluation runs. - - - `"auto"` - - - `"auto"` - - - `number` - - - `eval_samples: optional "auto" or number` - - Number of evaluation samples to generate per training step. - - - `"auto"` - - - `"auto"` - - - `number` - - - `learning_rate_multiplier: optional "auto" or number` - - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - - `"auto"` - - - `"auto"` - - - `number` - - - `n_epochs: optional "auto" or number` - - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - - `"auto"` - - - `"auto"` - - - `number` - - - `reasoning_effort: optional "default" or "low" or "medium" or "high"` - - Level of reasoning effort. - - - `"default"` - - - `"low"` - - - `"medium"` - - - `"high"` - - - `supervised: optional SupervisedMethod` - - Configuration for the supervised fine-tuning method. - - - `hyperparameters: optional SupervisedHyperparameters` - - The hyperparameters used for the fine-tuning job. - - - `batch_size: optional "auto" or number` - - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - - `"auto"` - - - `"auto"` - - - `number` - - - `learning_rate_multiplier: optional "auto" or number` - - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - - `"auto"` - - - `"auto"` - - - `number` - - - `n_epochs: optional "auto" or number` - - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - - `"auto"` - - - `"auto"` - - - `number` - -- `has_more: boolean` - -- `object: "list"` - - - `"list"` - -### Example - -```http -curl https://api.openai.com/v1/fine_tuning/jobs \ - -H "Authorization: Bearer $OPENAI_API_KEY" -``` - -#### Response - -```json -{ - "data": [ - { - "id": "id", - "created_at": 0, - "error": { - "code": "code", - "message": "message", - "param": "param" - }, - "fine_tuned_model": "fine_tuned_model", - "finished_at": 0, - "hyperparameters": { - "batch_size": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto" - }, - "model": "model", - "object": "fine_tuning.job", - "organization_id": "organization_id", - "result_files": [ - "file-abc123" - ], - "seed": 0, - "status": "validating_files", - "trained_tokens": 0, - "training_file": "training_file", - "validation_file": "validation_file", - "estimated_finish": 0, - "integrations": [ - { - "type": "wandb", - "wandb": { - "project": "my-wandb-project", - "entity": "entity", - "name": "name", - "tags": [ - "custom-tag" - ] - } - } - ], - "metadata": { - "foo": "string" - }, - "method": { - "type": "supervised", - "dpo": { - "hyperparameters": { - "batch_size": "auto", - "beta": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto" - } - }, - "reinforcement": { - "grader": { - "input": "input", - "name": "name", - "operation": "eq", - "reference": "reference", - "type": "string_check" - }, - "hyperparameters": { - "batch_size": "auto", - "compute_multiplier": "auto", - "eval_interval": "auto", - "eval_samples": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto", - "reasoning_effort": "default" - } - }, - "supervised": { - "hyperparameters": { - "batch_size": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto" - } - } - } - } - ], - "has_more": true, - "object": "list" -} -``` - -### Example - -```http -curl https://api.openai.com/v1/fine_tuning/jobs?limit=2&metadata[key]=value \ - -H "Authorization: Bearer $OPENAI_API_KEY" -``` - -#### Response - -```json -{ - "object": "list", - "data": [ - { - "object": "fine_tuning.job", - "id": "ftjob-abc123", - "model": "gpt-4o-mini-2024-07-18", - "created_at": 1721764800, - "fine_tuned_model": null, - "organization_id": "org-123", - "result_files": [], - "status": "queued", - "validation_file": null, - "training_file": "file-abc123", - "metadata": { - "key": "value" - } - }, - { ... }, - { ... } - ], "has_more": true -} -``` - -## Retrieve fine-tuning job - -**get** `/fine_tuning/jobs/{fine_tuning_job_id}` - -Get info about a fine-tuning job. - -[Learn more about fine-tuning](/docs/guides/model-optimization) - -### Path Parameters - -- `fine_tuning_job_id: string` - -### Returns - -- `FineTuningJob = object { id, created_at, error, 16 more }` - - The `fine_tuning.job` object represents a fine-tuning job that has been created through the API. - - - `id: string` - - The object identifier, which can be referenced in the API endpoints. - - - `created_at: number` - - The Unix timestamp (in seconds) for when the fine-tuning job was created. - - - `error: object { code, message, param }` - - For fine-tuning jobs that have `failed`, this will contain more information on the cause of the failure. - - - `code: string` - - A machine-readable error code. - - - `message: string` - - A human-readable error message. - - - `param: string` - - The parameter that was invalid, usually `training_file` or `validation_file`. This field will be null if the failure was not parameter-specific. - - - `fine_tuned_model: string` - - The name of the fine-tuned model that is being created. The value will be null if the fine-tuning job is still running. - - - `finished_at: number` - - The Unix timestamp (in seconds) for when the fine-tuning job was finished. The value will be null if the fine-tuning job is still running. - - - `hyperparameters: object { batch_size, learning_rate_multiplier, n_epochs }` - - The hyperparameters used for the fine-tuning job. This value will only be returned when running `supervised` jobs. - - - `batch_size: optional "auto" or number` - - Number of examples in each batch. A larger batch size means that model parameters - are updated less frequently, but with lower variance. - - - `"auto"` - - - `"auto"` - - - `number` - - - `learning_rate_multiplier: optional "auto" or number` - - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid - overfitting. - - - `"auto"` - - - `"auto"` - - - `number` - - - `n_epochs: optional "auto" or number` - - The number of epochs to train the model for. An epoch refers to one full cycle - through the training dataset. - - - `"auto"` - - - `"auto"` - - - `number` - - - `model: string` - - The base model that is being fine-tuned. - - - `object: "fine_tuning.job"` - - The object type, which is always "fine_tuning.job". - - - `"fine_tuning.job"` - - - `organization_id: string` - - The organization that owns the fine-tuning job. - - - `result_files: array of string` - - The compiled results file ID(s) for the fine-tuning job. You can retrieve the results with the [Files API](/docs/api-reference/files/retrieve-contents). - - - `seed: number` - - The seed used for the fine-tuning job. - - - `status: "validating_files" or "queued" or "running" or 3 more` - - The current status of the fine-tuning job, which can be either `validating_files`, `queued`, `running`, `succeeded`, `failed`, or `cancelled`. - - - `"validating_files"` - - - `"queued"` - - - `"running"` - - - `"succeeded"` - - - `"failed"` - - - `"cancelled"` - - - `trained_tokens: number` - - The total number of billable tokens processed by this fine-tuning job. The value will be null if the fine-tuning job is still running. - - - `training_file: string` - - The file ID used for training. You can retrieve the training data with the [Files API](/docs/api-reference/files/retrieve-contents). - - - `validation_file: string` - - The file ID used for validation. You can retrieve the validation results with the [Files API](/docs/api-reference/files/retrieve-contents). - - - `estimated_finish: optional number` - - The Unix timestamp (in seconds) for when the fine-tuning job is estimated to finish. The value will be null if the fine-tuning job is not running. - - - `integrations: optional array of FineTuningJobWandbIntegrationObject` - - A list of integrations to enable for this fine-tuning job. - - - `type: "wandb"` - - The type of the integration being enabled for the fine-tuning job - - - `"wandb"` - - - `wandb: FineTuningJobWandbIntegration` - - The settings for your integration with Weights and Biases. This payload specifies the project that - metrics will be sent to. Optionally, you can set an explicit display name for your run, add tags - to your run, and set a default entity (team, username, etc) to be associated with your run. - - - `project: string` - - The name of the project that the new run will be created under. - - - `entity: optional string` - - The entity to use for the run. This allows you to set the team or username of the WandB user that you would - like associated with the run. If not set, the default entity for the registered WandB API key is used. - - - `name: optional string` - - A display name to set for the run. If not set, we will use the Job ID as the name. - - - `tags: optional array of string` - - A list of tags to be attached to the newly created run. These tags are passed through directly to WandB. Some - default tags are generated by OpenAI: "openai/finetune", "openai/{base-model}", "openai/{ftjob-abcdef}". - - - `metadata: optional Metadata` - - Set of 16 key-value pairs that can be attached to an object. This can be - useful for storing additional information about the object in a structured - format, and querying for objects via API or the dashboard. - - Keys are strings with a maximum length of 64 characters. Values are strings - with a maximum length of 512 characters. - - - `method: optional object { type, dpo, reinforcement, supervised }` - - The method used for fine-tuning. - - - `type: "supervised" or "dpo" or "reinforcement"` - - The type of method. Is either `supervised`, `dpo`, or `reinforcement`. - - - `"supervised"` - - - `"dpo"` - - - `"reinforcement"` - - - `dpo: optional DpoMethod` - - Configuration for the DPO fine-tuning method. - - - `hyperparameters: optional DpoHyperparameters` - - The hyperparameters used for the DPO fine-tuning job. - - - `batch_size: optional "auto" or number` - - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - - `"auto"` - - - `"auto"` - - - `number` - - - `beta: optional "auto" or number` - - The beta value for the DPO method. A higher beta value will increase the weight of the penalty between the policy and reference model. - - - `"auto"` - - - `"auto"` - - - `number` - - - `learning_rate_multiplier: optional "auto" or number` - - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - - `"auto"` - - - `"auto"` - - - `number` - - - `n_epochs: optional "auto" or number` - - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - - `"auto"` - - - `"auto"` - - - `number` - - - `reinforcement: optional ReinforcementMethod` - - Configuration for the reinforcement fine-tuning method. - - - `grader: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - - The grader used for the fine-tuning job. - - - `StringCheckGrader = object { input, name, operation, 2 more }` - - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - - `input: string` - - The input text. This may include template strings. - - - `name: string` - - The name of the grader. - - - `operation: "eq" or "ne" or "like" or "ilike"` - - The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. - - - `"eq"` - - - `"ne"` - - - `"like"` - - - `"ilike"` - - - `reference: string` - - The reference text. This may include template strings. - - - `type: "string_check"` - - The object type, which is always `string_check`. - - - `"string_check"` - - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` - - A TextSimilarityGrader object which grades text based on similarity metrics. - - - `evaluation_metric: "cosine" or "fuzzy_match" or "bleu" or 8 more` - - The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, - `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, - or `rouge_l`. - - - `"cosine"` - - - `"fuzzy_match"` - - - `"bleu"` - - - `"gleu"` - - - `"meteor"` - - - `"rouge_1"` - - - `"rouge_2"` - - - `"rouge_3"` - - - `"rouge_4"` - - - `"rouge_5"` - - - `"rouge_l"` - - - `input: string` - - The text being graded. - - - `name: string` - - The name of the grader. - - - `reference: string` - - The text being graded against. - - - `type: "text_similarity"` - - The type of grader. - - - `"text_similarity"` - - - `PythonGrader = object { name, source, type, image_tag }` - - A PythonGrader object that runs a python script on the input. - - - `name: string` - - The name of the grader. - - - `source: string` - - The source code of the python script. - - - `type: "python"` - - The object type, which is always `python`. - - - `"python"` - - - `image_tag: optional string` - - The image tag to use for the python script. - - - `ScoreModelGrader = object { input, model, name, 3 more }` - - A ScoreModelGrader object that uses a model to assign a score to the input. - - - `input: array of object { content, role, type }` - - The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. - - - `content: string or ResponseInputText or object { text, type } or 3 more` - - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `role: "user" or "assistant" or "system" or "developer"` - - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. - - - `"user"` - - - `"assistant"` - - - `"system"` - - - `"developer"` - - - `type: optional "message"` - - The type of the message input. Always `message`. - - - `"message"` - - - `model: string` - - The model to use for the evaluation. - - - `name: string` - - The name of the grader. - - - `type: "score_model"` - - The object type, which is always `score_model`. - - - `"score_model"` - - - `range: optional array of number` - - The range of the score. Defaults to `[0, 1]`. - - - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` - - The sampling parameters for the model. - - - `max_completions_tokens: optional number` - - The maximum number of tokens the grader model may generate in its response. - - - `reasoning_effort: optional ReasoningEffort` - - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. - - - `"none"` - - - `"minimal"` - - - `"low"` - - - `"medium"` - - - `"high"` - - - `"xhigh"` - - - `seed: optional number` - - A seed value to initialize the randomness, during sampling. - - - `temperature: optional number` - - A higher temperature increases randomness in the outputs. - - - `top_p: optional number` - - An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - - `MultiGrader = object { calculate_output, graders, name, type }` - - A MultiGrader object combines the output of multiple graders to produce a single score. - - - `calculate_output: string` - - A formula to calculate the output based on grader results. - - - `graders: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - - `StringCheckGrader = object { input, name, operation, 2 more }` - - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - - `input: string` - - The input text. This may include template strings. - - - `name: string` - - The name of the grader. - - - `operation: "eq" or "ne" or "like" or "ilike"` - - The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. - - - `"eq"` - - - `"ne"` - - - `"like"` - - - `"ilike"` - - - `reference: string` - - The reference text. This may include template strings. - - - `type: "string_check"` - - The object type, which is always `string_check`. - - - `"string_check"` - - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` - - A TextSimilarityGrader object which grades text based on similarity metrics. - - - `evaluation_metric: "cosine" or "fuzzy_match" or "bleu" or 8 more` - - The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, - `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, - or `rouge_l`. - - - `"cosine"` - - - `"fuzzy_match"` - - - `"bleu"` - - - `"gleu"` - - - `"meteor"` - - - `"rouge_1"` - - - `"rouge_2"` - - - `"rouge_3"` - - - `"rouge_4"` - - - `"rouge_5"` - - - `"rouge_l"` - - - `input: string` - - The text being graded. - - - `name: string` - - The name of the grader. - - - `reference: string` - - The text being graded against. - - - `type: "text_similarity"` - - The type of grader. - - - `"text_similarity"` - - - `PythonGrader = object { name, source, type, image_tag }` - - A PythonGrader object that runs a python script on the input. - - - `name: string` - - The name of the grader. - - - `source: string` - - The source code of the python script. - - - `type: "python"` - - The object type, which is always `python`. - - - `"python"` - - - `image_tag: optional string` - - The image tag to use for the python script. - - - `ScoreModelGrader = object { input, model, name, 3 more }` - - A ScoreModelGrader object that uses a model to assign a score to the input. - - - `input: array of object { content, role, type }` - - The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. - - - `content: string or ResponseInputText or object { text, type } or 3 more` - - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `role: "user" or "assistant" or "system" or "developer"` - - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. - - - `"user"` - - - `"assistant"` - - - `"system"` - - - `"developer"` - - - `type: optional "message"` - - The type of the message input. Always `message`. - - - `"message"` - - - `model: string` - - The model to use for the evaluation. - - - `name: string` - - The name of the grader. - - - `type: "score_model"` - - The object type, which is always `score_model`. - - - `"score_model"` - - - `range: optional array of number` - - The range of the score. Defaults to `[0, 1]`. - - - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` - - The sampling parameters for the model. - - - `max_completions_tokens: optional number` - - The maximum number of tokens the grader model may generate in its response. - - - `reasoning_effort: optional ReasoningEffort` - - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. - - - `"none"` - - - `"minimal"` - - - `"low"` - - - `"medium"` - - - `"high"` - - - `"xhigh"` - - - `seed: optional number` - - A seed value to initialize the randomness, during sampling. - - - `temperature: optional number` - - A higher temperature increases randomness in the outputs. - - - `top_p: optional number` - - An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - - `LabelModelGrader = object { input, labels, model, 3 more }` - - A LabelModelGrader object which uses a model to assign labels to each item - in the evaluation. - - - `input: array of object { content, role, type }` - - - `content: string or ResponseInputText or object { text, type } or 3 more` - - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `role: "user" or "assistant" or "system" or "developer"` - - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. - - - `"user"` - - - `"assistant"` - - - `"system"` - - - `"developer"` - - - `type: optional "message"` - - The type of the message input. Always `message`. - - - `"message"` - - - `labels: array of string` - - The labels to assign to each item in the evaluation. - - - `model: string` - - The model to use for the evaluation. Must support structured outputs. - - - `name: string` - - The name of the grader. - - - `passing_labels: array of string` - - The labels that indicate a passing result. Must be a subset of labels. - - - `type: "label_model"` - - The object type, which is always `label_model`. - - - `"label_model"` - - - `name: string` - - The name of the grader. - - - `type: "multi"` - - The object type, which is always `multi`. - - - `"multi"` - - - `hyperparameters: optional ReinforcementHyperparameters` - - The hyperparameters used for the reinforcement fine-tuning job. - - - `batch_size: optional "auto" or number` - - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - - `"auto"` - - - `"auto"` - - - `number` - - - `compute_multiplier: optional "auto" or number` - - Multiplier on amount of compute used for exploring search space during training. - - - `"auto"` - - - `"auto"` - - - `number` - - - `eval_interval: optional "auto" or number` - - The number of training steps between evaluation runs. - - - `"auto"` - - - `"auto"` - - - `number` - - - `eval_samples: optional "auto" or number` - - Number of evaluation samples to generate per training step. - - - `"auto"` - - - `"auto"` - - - `number` - - - `learning_rate_multiplier: optional "auto" or number` - - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - - `"auto"` - - - `"auto"` - - - `number` - - - `n_epochs: optional "auto" or number` - - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - - `"auto"` - - - `"auto"` - - - `number` - - - `reasoning_effort: optional "default" or "low" or "medium" or "high"` - - Level of reasoning effort. - - - `"default"` - - - `"low"` - - - `"medium"` - - - `"high"` - - - `supervised: optional SupervisedMethod` - - Configuration for the supervised fine-tuning method. - - - `hyperparameters: optional SupervisedHyperparameters` - - The hyperparameters used for the fine-tuning job. - - - `batch_size: optional "auto" or number` - - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - - `"auto"` - - - `"auto"` - - - `number` - - - `learning_rate_multiplier: optional "auto" or number` - - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - - `"auto"` - - - `"auto"` - - - `number` - - - `n_epochs: optional "auto" or number` - - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - - `"auto"` - - - `"auto"` - - - `number` - -### Example - -```http -curl https://api.openai.com/v1/fine_tuning/jobs/$FINE_TUNING_JOB_ID \ - -H "Authorization: Bearer $OPENAI_API_KEY" -``` - -#### Response - -```json -{ - "id": "id", - "created_at": 0, - "error": { - "code": "code", - "message": "message", - "param": "param" - }, - "fine_tuned_model": "fine_tuned_model", - "finished_at": 0, - "hyperparameters": { - "batch_size": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto" - }, - "model": "model", - "object": "fine_tuning.job", - "organization_id": "organization_id", - "result_files": [ - "file-abc123" - ], - "seed": 0, - "status": "validating_files", - "trained_tokens": 0, - "training_file": "training_file", - "validation_file": "validation_file", - "estimated_finish": 0, - "integrations": [ - { - "type": "wandb", - "wandb": { - "project": "my-wandb-project", - "entity": "entity", - "name": "name", - "tags": [ - "custom-tag" - ] - } - } - ], - "metadata": { - "foo": "string" - }, - "method": { - "type": "supervised", - "dpo": { - "hyperparameters": { - "batch_size": "auto", - "beta": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto" - } - }, - "reinforcement": { - "grader": { - "input": "input", - "name": "name", - "operation": "eq", - "reference": "reference", - "type": "string_check" - }, - "hyperparameters": { - "batch_size": "auto", - "compute_multiplier": "auto", - "eval_interval": "auto", - "eval_samples": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto", - "reasoning_effort": "default" - } - }, - "supervised": { - "hyperparameters": { - "batch_size": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto" - } - } - } -} -``` - -### Example - -```http -curl https://api.openai.com/v1/fine_tuning/jobs/ft-AF1WoRqd3aJAHsqc9NY7iL8F \ - -H "Authorization: Bearer $OPENAI_API_KEY" -``` - -#### Response - -```json -{ - "object": "fine_tuning.job", - "id": "ftjob-abc123", - "model": "davinci-002", - "created_at": 1692661014, - "finished_at": 1692661190, - "fine_tuned_model": "ft:davinci-002:my-org:custom_suffix:7q8mpxmy", - "organization_id": "org-123", - "result_files": [ - "file-abc123" - ], - "status": "succeeded", - "validation_file": null, - "training_file": "file-abc123", - "hyperparameters": { - "n_epochs": 4, - "batch_size": 1, - "learning_rate_multiplier": 1.0 - }, - "trained_tokens": 5768, - "integrations": [], - "seed": 0, - "estimated_finish": 0, - "method": { - "type": "supervised", - "supervised": { - "hyperparameters": { - "n_epochs": 4, - "batch_size": 1, - "learning_rate_multiplier": 1.0 - } - } - } -} -``` - -## List fine-tuning events - -**get** `/fine_tuning/jobs/{fine_tuning_job_id}/events` - -Get status updates for a fine-tuning job. - -### Path Parameters - -- `fine_tuning_job_id: string` - -### Query Parameters - -- `after: optional string` - - Identifier for the last event from the previous pagination request. - -- `limit: optional number` - - Number of events to retrieve. - -### Returns - -- `data: array of FineTuningJobEvent` - - - `id: string` - - The object identifier. - - - `created_at: number` - - The Unix timestamp (in seconds) for when the fine-tuning job was created. - - - `level: "info" or "warn" or "error"` - - The log level of the event. - - - `"info"` - - - `"warn"` - - - `"error"` - - - `message: string` - - The message of the event. - - - `object: "fine_tuning.job.event"` - - The object type, which is always "fine_tuning.job.event". - - - `"fine_tuning.job.event"` - - - `data: optional unknown` - - The data associated with the event. - - - `type: optional "message" or "metrics"` - - The type of event. - - - `"message"` - - - `"metrics"` - -- `has_more: boolean` - -- `object: "list"` - - - `"list"` - -### Example - -```http -curl https://api.openai.com/v1/fine_tuning/jobs/$FINE_TUNING_JOB_ID/events \ - -H "Authorization: Bearer $OPENAI_API_KEY" -``` - -#### Response - -```json -{ - "data": [ - { - "id": "id", - "created_at": 0, - "level": "info", - "message": "message", - "object": "fine_tuning.job.event", - "data": {}, - "type": "message" - } - ], - "has_more": true, - "object": "list" -} -``` - -### Example - -```http -curl https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/events \ - -H "Authorization: Bearer $OPENAI_API_KEY" -``` - -#### Response - -```json -{ - "object": "list", - "data": [ - { - "object": "fine_tuning.job.event", - "id": "ft-event-ddTJfwuMVpfLXseO0Am0Gqjm", - "created_at": 1721764800, - "level": "info", - "message": "Fine tuning job successfully completed", - "data": null, - "type": "message" - }, - { - "object": "fine_tuning.job.event", - "id": "ft-event-tyiGuB72evQncpH87xe505Sv", - "created_at": 1721764800, - "level": "info", - "message": "New fine-tuned model created: ft:gpt-4o-mini:openai::7p4lURel", - "data": null, - "type": "message" - } - ], - "has_more": true -} -``` - -## Cancel fine-tuning - -**post** `/fine_tuning/jobs/{fine_tuning_job_id}/cancel` - -Immediately cancel a fine-tune job. - -### Path Parameters - -- `fine_tuning_job_id: string` - -### Returns - -- `FineTuningJob = object { id, created_at, error, 16 more }` - - The `fine_tuning.job` object represents a fine-tuning job that has been created through the API. - - - `id: string` - - The object identifier, which can be referenced in the API endpoints. - - - `created_at: number` - - The Unix timestamp (in seconds) for when the fine-tuning job was created. - - - `error: object { code, message, param }` - - For fine-tuning jobs that have `failed`, this will contain more information on the cause of the failure. - - - `code: string` - - A machine-readable error code. - - - `message: string` - - A human-readable error message. - - - `param: string` - - The parameter that was invalid, usually `training_file` or `validation_file`. This field will be null if the failure was not parameter-specific. - - - `fine_tuned_model: string` - - The name of the fine-tuned model that is being created. The value will be null if the fine-tuning job is still running. - - - `finished_at: number` - - The Unix timestamp (in seconds) for when the fine-tuning job was finished. The value will be null if the fine-tuning job is still running. - - - `hyperparameters: object { batch_size, learning_rate_multiplier, n_epochs }` - - The hyperparameters used for the fine-tuning job. This value will only be returned when running `supervised` jobs. - - - `batch_size: optional "auto" or number` - - Number of examples in each batch. A larger batch size means that model parameters - are updated less frequently, but with lower variance. - - - `"auto"` - - - `"auto"` - - - `number` - - - `learning_rate_multiplier: optional "auto" or number` - - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid - overfitting. - - - `"auto"` - - - `"auto"` - - - `number` - - - `n_epochs: optional "auto" or number` - - The number of epochs to train the model for. An epoch refers to one full cycle - through the training dataset. - - - `"auto"` - - - `"auto"` - - - `number` - - - `model: string` - - The base model that is being fine-tuned. - - - `object: "fine_tuning.job"` - - The object type, which is always "fine_tuning.job". - - - `"fine_tuning.job"` - - - `organization_id: string` - - The organization that owns the fine-tuning job. - - - `result_files: array of string` - - The compiled results file ID(s) for the fine-tuning job. You can retrieve the results with the [Files API](/docs/api-reference/files/retrieve-contents). - - - `seed: number` - - The seed used for the fine-tuning job. - - - `status: "validating_files" or "queued" or "running" or 3 more` - - The current status of the fine-tuning job, which can be either `validating_files`, `queued`, `running`, `succeeded`, `failed`, or `cancelled`. - - - `"validating_files"` - - - `"queued"` - - - `"running"` - - - `"succeeded"` - - - `"failed"` - - - `"cancelled"` - - - `trained_tokens: number` - - The total number of billable tokens processed by this fine-tuning job. The value will be null if the fine-tuning job is still running. - - - `training_file: string` - - The file ID used for training. You can retrieve the training data with the [Files API](/docs/api-reference/files/retrieve-contents). - - - `validation_file: string` - - The file ID used for validation. You can retrieve the validation results with the [Files API](/docs/api-reference/files/retrieve-contents). - - - `estimated_finish: optional number` - - The Unix timestamp (in seconds) for when the fine-tuning job is estimated to finish. The value will be null if the fine-tuning job is not running. - - - `integrations: optional array of FineTuningJobWandbIntegrationObject` - - A list of integrations to enable for this fine-tuning job. - - - `type: "wandb"` - - The type of the integration being enabled for the fine-tuning job - - - `"wandb"` - - - `wandb: FineTuningJobWandbIntegration` - - The settings for your integration with Weights and Biases. This payload specifies the project that - metrics will be sent to. Optionally, you can set an explicit display name for your run, add tags - to your run, and set a default entity (team, username, etc) to be associated with your run. - - - `project: string` - - The name of the project that the new run will be created under. - - - `entity: optional string` - - The entity to use for the run. This allows you to set the team or username of the WandB user that you would - like associated with the run. If not set, the default entity for the registered WandB API key is used. - - - `name: optional string` - - A display name to set for the run. If not set, we will use the Job ID as the name. - - - `tags: optional array of string` - - A list of tags to be attached to the newly created run. These tags are passed through directly to WandB. Some - default tags are generated by OpenAI: "openai/finetune", "openai/{base-model}", "openai/{ftjob-abcdef}". - - - `metadata: optional Metadata` - - Set of 16 key-value pairs that can be attached to an object. This can be - useful for storing additional information about the object in a structured - format, and querying for objects via API or the dashboard. - - Keys are strings with a maximum length of 64 characters. Values are strings - with a maximum length of 512 characters. - - - `method: optional object { type, dpo, reinforcement, supervised }` - - The method used for fine-tuning. - - - `type: "supervised" or "dpo" or "reinforcement"` - - The type of method. Is either `supervised`, `dpo`, or `reinforcement`. - - - `"supervised"` - - - `"dpo"` - - - `"reinforcement"` - - - `dpo: optional DpoMethod` - - Configuration for the DPO fine-tuning method. - - - `hyperparameters: optional DpoHyperparameters` - - The hyperparameters used for the DPO fine-tuning job. - - - `batch_size: optional "auto" or number` - - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - - `"auto"` - - - `"auto"` - - - `number` - - - `beta: optional "auto" or number` - - The beta value for the DPO method. A higher beta value will increase the weight of the penalty between the policy and reference model. - - - `"auto"` - - - `"auto"` - - - `number` - - - `learning_rate_multiplier: optional "auto" or number` - - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - - `"auto"` - - - `"auto"` - - - `number` - - - `n_epochs: optional "auto" or number` - - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - - `"auto"` - - - `"auto"` - - - `number` - - - `reinforcement: optional ReinforcementMethod` - - Configuration for the reinforcement fine-tuning method. - - - `grader: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - - The grader used for the fine-tuning job. - - - `StringCheckGrader = object { input, name, operation, 2 more }` - - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - - `input: string` - - The input text. This may include template strings. - - - `name: string` - - The name of the grader. - - - `operation: "eq" or "ne" or "like" or "ilike"` - - The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. - - - `"eq"` - - - `"ne"` - - - `"like"` - - - `"ilike"` - - - `reference: string` - - The reference text. This may include template strings. - - - `type: "string_check"` - - The object type, which is always `string_check`. - - - `"string_check"` - - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` - - A TextSimilarityGrader object which grades text based on similarity metrics. - - - `evaluation_metric: "cosine" or "fuzzy_match" or "bleu" or 8 more` - - The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, - `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, - or `rouge_l`. - - - `"cosine"` - - - `"fuzzy_match"` - - - `"bleu"` - - - `"gleu"` - - - `"meteor"` - - - `"rouge_1"` - - - `"rouge_2"` - - - `"rouge_3"` - - - `"rouge_4"` - - - `"rouge_5"` - - - `"rouge_l"` - - - `input: string` - - The text being graded. - - - `name: string` - - The name of the grader. - - - `reference: string` - - The text being graded against. - - - `type: "text_similarity"` - - The type of grader. - - - `"text_similarity"` - - - `PythonGrader = object { name, source, type, image_tag }` - - A PythonGrader object that runs a python script on the input. - - - `name: string` - - The name of the grader. - - - `source: string` - - The source code of the python script. - - - `type: "python"` - - The object type, which is always `python`. - - - `"python"` - - - `image_tag: optional string` - - The image tag to use for the python script. - - - `ScoreModelGrader = object { input, model, name, 3 more }` - - A ScoreModelGrader object that uses a model to assign a score to the input. - - - `input: array of object { content, role, type }` - - The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. - - - `content: string or ResponseInputText or object { text, type } or 3 more` - - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `role: "user" or "assistant" or "system" or "developer"` - - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. - - - `"user"` - - - `"assistant"` - - - `"system"` - - - `"developer"` - - - `type: optional "message"` - - The type of the message input. Always `message`. - - - `"message"` - - - `model: string` - - The model to use for the evaluation. - - - `name: string` - - The name of the grader. - - - `type: "score_model"` - - The object type, which is always `score_model`. - - - `"score_model"` - - - `range: optional array of number` - - The range of the score. Defaults to `[0, 1]`. - - - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` - - The sampling parameters for the model. - - - `max_completions_tokens: optional number` - - The maximum number of tokens the grader model may generate in its response. - - - `reasoning_effort: optional ReasoningEffort` - - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. - - - `"none"` - - - `"minimal"` - - - `"low"` - - - `"medium"` - - - `"high"` - - - `"xhigh"` - - - `seed: optional number` - - A seed value to initialize the randomness, during sampling. - - - `temperature: optional number` - - A higher temperature increases randomness in the outputs. - - - `top_p: optional number` - - An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - - `MultiGrader = object { calculate_output, graders, name, type }` - - A MultiGrader object combines the output of multiple graders to produce a single score. - - - `calculate_output: string` - - A formula to calculate the output based on grader results. - - - `graders: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - - `StringCheckGrader = object { input, name, operation, 2 more }` - - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - - `input: string` - - The input text. This may include template strings. - - - `name: string` - - The name of the grader. - - - `operation: "eq" or "ne" or "like" or "ilike"` - - The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. - - - `"eq"` - - - `"ne"` - - - `"like"` - - - `"ilike"` - - - `reference: string` - - The reference text. This may include template strings. - - - `type: "string_check"` - - The object type, which is always `string_check`. - - - `"string_check"` - - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` - - A TextSimilarityGrader object which grades text based on similarity metrics. - - - `evaluation_metric: "cosine" or "fuzzy_match" or "bleu" or 8 more` - - The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, - `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, - or `rouge_l`. - - - `"cosine"` - - - `"fuzzy_match"` - - - `"bleu"` - - - `"gleu"` - - - `"meteor"` - - - `"rouge_1"` - - - `"rouge_2"` - - - `"rouge_3"` - - - `"rouge_4"` - - - `"rouge_5"` - - - `"rouge_l"` - - - `input: string` - - The text being graded. - - - `name: string` - - The name of the grader. - - - `reference: string` - - The text being graded against. - - - `type: "text_similarity"` - - The type of grader. - - - `"text_similarity"` - - - `PythonGrader = object { name, source, type, image_tag }` - - A PythonGrader object that runs a python script on the input. - - - `name: string` - - The name of the grader. - - - `source: string` - - The source code of the python script. - - - `type: "python"` - - The object type, which is always `python`. - - - `"python"` - - - `image_tag: optional string` - - The image tag to use for the python script. - - - `ScoreModelGrader = object { input, model, name, 3 more }` - - A ScoreModelGrader object that uses a model to assign a score to the input. - - - `input: array of object { content, role, type }` - - The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. - - - `content: string or ResponseInputText or object { text, type } or 3 more` - - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `role: "user" or "assistant" or "system" or "developer"` - - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. - - - `"user"` - - - `"assistant"` - - - `"system"` - - - `"developer"` - - - `type: optional "message"` - - The type of the message input. Always `message`. - - - `"message"` - - - `model: string` - - The model to use for the evaluation. - - - `name: string` - - The name of the grader. - - - `type: "score_model"` - - The object type, which is always `score_model`. - - - `"score_model"` - - - `range: optional array of number` - - The range of the score. Defaults to `[0, 1]`. - - - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` - - The sampling parameters for the model. - - - `max_completions_tokens: optional number` - - The maximum number of tokens the grader model may generate in its response. - - - `reasoning_effort: optional ReasoningEffort` - - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. - - - `"none"` - - - `"minimal"` - - - `"low"` - - - `"medium"` - - - `"high"` - - - `"xhigh"` - - - `seed: optional number` - - A seed value to initialize the randomness, during sampling. - - - `temperature: optional number` - - A higher temperature increases randomness in the outputs. - - - `top_p: optional number` - - An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - - `LabelModelGrader = object { input, labels, model, 3 more }` - - A LabelModelGrader object which uses a model to assign labels to each item - in the evaluation. - - - `input: array of object { content, role, type }` - - - `content: string or ResponseInputText or object { text, type } or 3 more` - - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` - - The type of the image input. Always `input_image`. - - - `"input_image"` - - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - - `role: "user" or "assistant" or "system" or "developer"` - - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. - - - `"user"` - - - `"assistant"` - - - `"system"` - - - `"developer"` - - - `type: optional "message"` - - The type of the message input. Always `message`. - - - `"message"` - - - `labels: array of string` - - The labels to assign to each item in the evaluation. - - - `model: string` - - The model to use for the evaluation. Must support structured outputs. - - - `name: string` - - The name of the grader. - - - `passing_labels: array of string` - - The labels that indicate a passing result. Must be a subset of labels. - - - `type: "label_model"` - - The object type, which is always `label_model`. - - - `"label_model"` - - - `name: string` - - The name of the grader. - - - `type: "multi"` - - The object type, which is always `multi`. - - - `"multi"` - - - `hyperparameters: optional ReinforcementHyperparameters` - - The hyperparameters used for the reinforcement fine-tuning job. - - - `batch_size: optional "auto" or number` - - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - - `"auto"` - - - `"auto"` - - - `number` - - - `compute_multiplier: optional "auto" or number` - - Multiplier on amount of compute used for exploring search space during training. - - - `"auto"` - - - `"auto"` - - - `number` - - - `eval_interval: optional "auto" or number` - - The number of training steps between evaluation runs. - - - `"auto"` - - - `"auto"` - - - `number` - - - `eval_samples: optional "auto" or number` - - Number of evaluation samples to generate per training step. - - - `"auto"` - - - `"auto"` - - - `number` - - - `learning_rate_multiplier: optional "auto" or number` - - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - - `"auto"` - - - `"auto"` - - - `number` - - - `n_epochs: optional "auto" or number` - - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - - `"auto"` - - - `"auto"` - - - `number` - - - `reasoning_effort: optional "default" or "low" or "medium" or "high"` - - Level of reasoning effort. - - - `"default"` - - - `"low"` - - - `"medium"` - - - `"high"` - - - `supervised: optional SupervisedMethod` - - Configuration for the supervised fine-tuning method. - - - `hyperparameters: optional SupervisedHyperparameters` - - The hyperparameters used for the fine-tuning job. - - - `batch_size: optional "auto" or number` - - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - - `"auto"` +```json +{ + "id": "id", + "created_at": 0, + "error": { + "code": "code", + "message": "message", + "param": "param" + }, + "fine_tuned_model": "fine_tuned_model", + "finished_at": 0, + "hyperparameters": { + "batch_size": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto" + }, + "model": "model", + "object": "fine_tuning.job", + "organization_id": "organization_id", + "result_files": [ + "file-abc123" + ], + "seed": 0, + "status": "validating_files", + "trained_tokens": 0, + "training_file": "training_file", + "validation_file": "validation_file", + "estimated_finish": 0, + "integrations": [ + { + "type": "wandb", + "wandb": { + "project": "my-wandb-project", + "entity": "entity", + "name": "name", + "tags": [ + "custom-tag" + ] + } + } + ], + "metadata": { + "foo": "string" + }, + "method": { + "type": "supervised", + "dpo": { + "hyperparameters": { + "batch_size": "auto", + "beta": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto" + } + }, + "reinforcement": { + "grader": { + "input": "input", + "name": "name", + "operation": "eq", + "reference": "reference", + "type": "string_check" + }, + "hyperparameters": { + "batch_size": "auto", + "compute_multiplier": "auto", + "eval_interval": "auto", + "eval_samples": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto", + "reasoning_effort": "default" + } + }, + "supervised": { + "hyperparameters": { + "batch_size": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto" + } + } + } +} +``` - - `"auto"` +### Example - - `number` +```http +curl https://api.openai.com/v1/fine_tuning/jobs \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer $OPENAI_API_KEY" \ + -d '{ + "training_file": "file-BK7bzQj3FfZFXr7DbL6xJwfo", + "model": "gpt-4o-mini" + }' +``` - - `learning_rate_multiplier: optional "auto" or number` +#### Response - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. +```json +{ + "object": "fine_tuning.job", + "id": "ftjob-abc123", + "model": "gpt-4o-mini-2024-07-18", + "created_at": 1721764800, + "fine_tuned_model": null, + "organization_id": "org-123", + "result_files": [], + "status": "queued", + "validation_file": null, + "training_file": "file-abc123", + "method": { + "type": "supervised", + "supervised": { + "hyperparameters": { + "batch_size": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto", + } + } + }, + "metadata": null +} +``` - - `"auto"` +### Epochs - - `"auto"` +```http +curl https://api.openai.com/v1/fine_tuning/jobs \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer $OPENAI_API_KEY" \ + -d '{ + "training_file": "file-abc123", + "model": "gpt-4o-mini", + "method": { + "type": "supervised", + "supervised": { + "hyperparameters": { + "n_epochs": 2 + } + } + } + }' +``` - - `number` +#### Response - - `n_epochs: optional "auto" or number` +```json +{ + "object": "fine_tuning.job", + "id": "ftjob-abc123", + "model": "gpt-4o-mini", + "created_at": 1721764800, + "fine_tuned_model": null, + "organization_id": "org-123", + "result_files": [], + "status": "queued", + "validation_file": null, + "training_file": "file-abc123", + "hyperparameters": { + "batch_size": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": 2 + }, + "method": { + "type": "supervised", + "supervised": { + "hyperparameters": { + "batch_size": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": 2 + } + } + }, + "metadata": null, + "error": { + "code": null, + "message": null, + "param": null + }, + "finished_at": null, + "seed": 683058546, + "trained_tokens": null, + "estimated_finish": null, + "integrations": [], + "user_provided_suffix": null, + "usage_metrics": null, + "shared_with_openai": false +} +``` - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. +### DPO - - `"auto"` +```http +curl https://api.openai.com/v1/fine_tuning/jobs \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer $OPENAI_API_KEY" \ + -d '{ + "training_file": "file-abc123", + "validation_file": "file-abc123", + "model": "gpt-4o-mini", + "method": { + "type": "dpo", + "dpo": { + "hyperparameters": { + "beta": 0.1 + } + } + } + }' +``` - - `"auto"` +#### Response - - `number` +```json +{ + "object": "fine_tuning.job", + "id": "ftjob-abc", + "model": "gpt-4o-mini", + "created_at": 1746130590, + "fine_tuned_model": null, + "organization_id": "org-abc", + "result_files": [], + "status": "queued", + "validation_file": "file-123", + "training_file": "file-abc", + "method": { + "type": "dpo", + "dpo": { + "hyperparameters": { + "beta": 0.1, + "batch_size": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto" + } + } + }, + "metadata": null, + "error": { + "code": null, + "message": null, + "param": null + }, + "finished_at": null, + "hyperparameters": null, + "seed": 1036326793, + "estimated_finish": null, + "integrations": [], + "user_provided_suffix": null, + "usage_metrics": null, + "shared_with_openai": false +} +``` -### Example +### Reinforcement ```http -curl https://api.openai.com/v1/fine_tuning/jobs/$FINE_TUNING_JOB_ID/cancel \ - -X POST \ - -H "Authorization: Bearer $OPENAI_API_KEY" +curl https://api.openai.com/v1/fine_tuning/jobs \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer $OPENAI_API_KEY" \ + -d '{ + "training_file": "file-abc", + "validation_file": "file-123", + "model": "o4-mini", + "method": { + "type": "reinforcement", + "reinforcement": { + "grader": { + "type": "string_check", + "name": "Example string check grader", + "input": "{{sample.output_text}}", + "reference": "{{item.label}}", + "operation": "eq" + }, + "hyperparameters": { + "reasoning_effort": "medium" + } + } + } + }' ``` #### Response ```json { - "id": "id", - "created_at": 0, - "error": { - "code": "code", - "message": "message", - "param": "param" - }, - "fine_tuned_model": "fine_tuned_model", - "finished_at": 0, + "object": "fine_tuning.job", + "id": "ftjob-abc123", + "model": "o4-mini", + "created_at": 1721764800, + "finished_at": null, + "fine_tuned_model": null, + "organization_id": "org-123", + "result_files": [], + "status": "validating_files", + "validation_file": "file-123", + "training_file": "file-abc", + "trained_tokens": null, + "error": {}, + "user_provided_suffix": null, + "seed": 950189191, + "estimated_finish": null, + "integrations": [], + "method": { + "type": "reinforcement", + "reinforcement": { "hyperparameters": { "batch_size": "auto", "learning_rate_multiplier": "auto", - "n_epochs": "auto" + "n_epochs": "auto", + "eval_interval": "auto", + "eval_samples": "auto", + "compute_multiplier": "auto", + "reasoning_effort": "medium" }, - "model": "model", - "object": "fine_tuning.job", - "organization_id": "organization_id", - "result_files": [ - "file-abc123" - ], - "seed": 0, - "status": "validating_files", - "trained_tokens": 0, - "training_file": "training_file", - "validation_file": "validation_file", - "estimated_finish": 0, - "integrations": [ - { - "type": "wandb", - "wandb": { - "project": "my-wandb-project", - "entity": "entity", - "name": "name", - "tags": [ - "custom-tag" - ] - } + "grader": { + "type": "string_check", + "name": "Example string check grader", + "input": "{{sample.output_text}}", + "reference": "{{item.label}}", + "operation": "eq" + }, + "response_format": null } - ], - "metadata": { - "foo": "string" }, + "metadata": null, + "usage_metrics": null, + "shared_with_openai": false +} + +``` + +### Validation file + +```http +curl https://api.openai.com/v1/fine_tuning/jobs \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer $OPENAI_API_KEY" \ + -d '{ + "training_file": "file-abc123", + "validation_file": "file-abc123", + "model": "gpt-4o-mini" + }' +``` + +#### Response + +```json +{ + "object": "fine_tuning.job", + "id": "ftjob-abc123", + "model": "gpt-4o-mini-2024-07-18", + "created_at": 1721764800, + "fine_tuned_model": null, + "organization_id": "org-123", + "result_files": [], + "status": "queued", + "validation_file": "file-abc123", + "training_file": "file-abc123", "method": { "type": "supervised", - "dpo": { - "hyperparameters": { - "batch_size": "auto", - "beta": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto" - } - }, - "reinforcement": { - "grader": { - "input": "input", - "name": "name", - "operation": "eq", - "reference": "reference", - "type": "string_check" - }, - "hyperparameters": { - "batch_size": "auto", - "compute_multiplier": "auto", - "eval_interval": "auto", - "eval_samples": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto", - "reasoning_effort": "default" - } - }, "supervised": { "hyperparameters": { "batch_size": "auto", "learning_rate_multiplier": "auto", - "n_epochs": "auto" - } + "n_epochs": "auto", } } + }, + "metadata": null } ``` -### Example +### W&B Integration ```http -curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/cancel \ - -H "Authorization: Bearer $OPENAI_API_KEY" +curl https://api.openai.com/v1/fine_tuning/jobs \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer $OPENAI_API_KEY" \ + -d '{ + "training_file": "file-abc123", + "validation_file": "file-abc123", + "model": "gpt-4o-mini", + "integrations": [ + { + "type": "wandb", + "wandb": { + "project": "my-wandb-project", + "name": "ft-run-display-name" + "tags": [ + "first-experiment", "v2" + ] + } + } + ] + }' ``` #### Response @@ -8743,27 +2843,56 @@ curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/cancel \ "fine_tuned_model": null, "organization_id": "org-123", "result_files": [], - "status": "cancelled", + "status": "queued", "validation_file": "file-abc123", - "training_file": "file-abc123" + "training_file": "file-abc123", + "integrations": [ + { + "type": "wandb", + "wandb": { + "project": "my-wandb-project", + "entity": None, + "run_id": "ftjob-abc123" + } + } + ], + "method": { + "type": "supervised", + "supervised": { + "hyperparameters": { + "batch_size": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto", + } + } + }, + "metadata": null } ``` -## Pause fine-tuning +## List fine-tuning jobs -**post** `/fine_tuning/jobs/{fine_tuning_job_id}/pause` +**get** `/fine_tuning/jobs` -Pause a fine-tune job. +List your organization's fine-tuning jobs -### Path Parameters +### Query Parameters -- `fine_tuning_job_id: string` +- `after: optional string` -### Returns + Identifier for the last job from the previous pagination request. -- `FineTuningJob = object { id, created_at, error, 16 more }` +- `limit: optional number` - The `fine_tuning.job` object represents a fine-tuning job that has been created through the API. + Number of fine-tuning jobs to retrieve. + +- `metadata: optional map[string]` + + Optional metadata filter. To filter, use the syntax `metadata[k]=v`. Alternatively, set `metadata=null` to indicate no metadata. + +### Returns + +- `data: array of FineTuningJob` - `id: string` @@ -9001,7 +3130,7 @@ Pause a fine-tune job. The grader used for the fine-tuning job. - - `StringCheckGrader = object { input, name, operation, 2 more }` + - `StringCheckGrader object { input, name, operation, 2 more }` A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. @@ -9035,7 +3164,7 @@ Pause a fine-tune job. - `"string_check"` - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` A TextSimilarityGrader object which grades text based on similarity metrics. @@ -9085,7 +3214,7 @@ Pause a fine-tune job. - `"text_similarity"` - - `PythonGrader = object { name, source, type, image_tag }` + - `PythonGrader object { name, source, type, image_tag }` A PythonGrader object that runs a python script on the input. @@ -9107,7 +3236,7 @@ Pause a fine-tune job. The image tag to use for the python script. - - `ScoreModelGrader = object { input, model, name, 3 more }` + - `ScoreModelGrader object { input, model, name, 3 more }` A ScoreModelGrader object that uses a model to assign a score to the input. @@ -9123,7 +3252,7 @@ Pause a fine-tune job. A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. @@ -9137,7 +3266,7 @@ Pause a fine-tune job. - `"input_text"` - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -9151,7 +3280,7 @@ Pause a fine-tune job. - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -9169,7 +3298,7 @@ Pause a fine-tune job. The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. @@ -9203,21 +3332,11 @@ Pause a fine-tune job. A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -9231,7 +3350,7 @@ Pause a fine-tune job. - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -9249,31 +3368,10 @@ Pause a fine-tune job. The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or @@ -9356,7 +3454,7 @@ Pause a fine-tune job. An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `MultiGrader = object { calculate_output, graders, name, type }` + - `MultiGrader object { calculate_output, graders, name, type }` A MultiGrader object combines the output of multiple graders to produce a single score. @@ -9368,1016 +3466,1129 @@ Pause a fine-tune job. A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - `StringCheckGrader = object { input, name, operation, 2 more }` + - `StringCheckGrader object { input, name, operation, 2 more }` A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - `input: string` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` - The input text. This may include template strings. + A TextSimilarityGrader object which grades text based on similarity metrics. - - `name: string` + - `PythonGrader object { name, source, type, image_tag }` - The name of the grader. + A PythonGrader object that runs a python script on the input. - - `operation: "eq" or "ne" or "like" or "ilike"` + - `ScoreModelGrader object { input, model, name, 3 more }` - The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. + A ScoreModelGrader object that uses a model to assign a score to the input. - - `"eq"` + - `LabelModelGrader object { input, labels, model, 3 more }` - - `"ne"` + A LabelModelGrader object which uses a model to assign labels to each item + in the evaluation. - - `"like"` + - `input: array of object { content, role, type }` - - `"ilike"` + - `content: string or ResponseInputText or object { text, type } or 3 more` - - `reference: string` + Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - The reference text. This may include template strings. + - `TextInput = string` - - `type: "string_check"` + A text input to the model. - The object type, which is always `string_check`. + - `ResponseInputText object { text, type }` - - `"string_check"` + A text input to the model. - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` + - `OutputText object { text, type }` - A TextSimilarityGrader object which grades text based on similarity metrics. + A text output from the model. - - `evaluation_metric: "cosine" or "fuzzy_match" or "bleu" or 8 more` + - `text: string` - The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, - `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, - or `rouge_l`. + The text output from the model. - - `"cosine"` + - `type: "output_text"` - - `"fuzzy_match"` + The type of the output text. Always `output_text`. - - `"bleu"` + - `"output_text"` - - `"gleu"` + - `InputImage object { image_url, type, detail }` - - `"meteor"` + An image input block used within EvalItem content arrays. - - `"rouge_1"` + - `image_url: string` - - `"rouge_2"` + The URL of the image input. - - `"rouge_3"` + - `type: "input_image"` - - `"rouge_4"` + The type of the image input. Always `input_image`. - - `"rouge_5"` + - `"input_image"` - - `"rouge_l"` + - `detail: optional string` - - `input: string` + The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - The text being graded. + - `ResponseInputAudio object { input_audio, type }` - - `name: string` + An audio input to the model. - The name of the grader. + - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - `reference: string` + A list of inputs, each of which may be either an input text, output text, input + image, or input audio object. - The text being graded against. + - `role: "user" or "assistant" or "system" or "developer"` - - `type: "text_similarity"` + The role of the message input. One of `user`, `assistant`, `system`, or + `developer`. - The type of grader. + - `"user"` - - `"text_similarity"` + - `"assistant"` - - `PythonGrader = object { name, source, type, image_tag }` + - `"system"` - A PythonGrader object that runs a python script on the input. + - `"developer"` + + - `type: optional "message"` + + The type of the message input. Always `message`. + + - `"message"` + + - `labels: array of string` + + The labels to assign to each item in the evaluation. + + - `model: string` + + The model to use for the evaluation. Must support structured outputs. - `name: string` The name of the grader. - - `source: string` + - `passing_labels: array of string` - The source code of the python script. + The labels that indicate a passing result. Must be a subset of labels. - - `type: "python"` + - `type: "label_model"` - The object type, which is always `python`. + The object type, which is always `label_model`. - - `"python"` + - `"label_model"` - - `image_tag: optional string` + - `name: string` - The image tag to use for the python script. + The name of the grader. - - `ScoreModelGrader = object { input, model, name, 3 more }` + - `type: "multi"` - A ScoreModelGrader object that uses a model to assign a score to the input. + The object type, which is always `multi`. - - `input: array of object { content, role, type }` + - `"multi"` - The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. + - `hyperparameters: optional ReinforcementHyperparameters` + + The hyperparameters used for the reinforcement fine-tuning job. + + - `batch_size: optional "auto" or number` + + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - `content: string or ResponseInputText or object { text, type } or 3 more` + - `"auto"` - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. + - `"auto"` - - `TextInput = string` + - `number` - A text input to the model. + - `compute_multiplier: optional "auto" or number` - - `ResponseInputText = object { text, type }` + Multiplier on amount of compute used for exploring search space during training. - A text input to the model. + - `"auto"` - - `text: string` + - `"auto"` - The text input to the model. + - `number` - - `type: "input_text"` + - `eval_interval: optional "auto" or number` - The type of the input item. Always `input_text`. + The number of training steps between evaluation runs. - - `"input_text"` + - `"auto"` - - `OutputText = object { text, type }` + - `"auto"` - A text output from the model. + - `number` - - `text: string` + - `eval_samples: optional "auto" or number` - The text output from the model. + Number of evaluation samples to generate per training step. - - `type: "output_text"` + - `"auto"` - The type of the output text. Always `output_text`. + - `"auto"` - - `"output_text"` + - `number` - - `InputImage = object { image_url, type, detail }` + - `learning_rate_multiplier: optional "auto" or number` - An image input block used within EvalItem content arrays. + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - `image_url: string` + - `"auto"` - The URL of the image input. + - `"auto"` - - `type: "input_image"` + - `number` - The type of the image input. Always `input_image`. + - `n_epochs: optional "auto" or number` - - `"input_image"` + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - `detail: optional string` + - `"auto"` - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + - `"auto"` - - `ResponseInputAudio = object { input_audio, type }` + - `number` - An audio input to the model. + - `reasoning_effort: optional "default" or "low" or "medium" or "high"` - - `input_audio: object { data, format }` + Level of reasoning effort. - - `data: string` + - `"default"` - Base64-encoded audio data. + - `"low"` - - `format: "mp3" or "wav"` + - `"medium"` - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + - `"high"` - - `"mp3"` + - `supervised: optional SupervisedMethod` - - `"wav"` + Configuration for the supervised fine-tuning method. - - `type: "input_audio"` + - `hyperparameters: optional SupervisedHyperparameters` - The type of the input item. Always `input_audio`. + The hyperparameters used for the fine-tuning job. - - `"input_audio"` + - `batch_size: optional "auto" or number` - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. + - `"auto"` - - `TextInput = string` + - `"auto"` - A text input to the model. + - `number` - - `ResponseInputText = object { text, type }` + - `learning_rate_multiplier: optional "auto" or number` - A text input to the model. + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - `text: string` + - `"auto"` - The text input to the model. + - `"auto"` - - `type: "input_text"` + - `number` - The type of the input item. Always `input_text`. + - `n_epochs: optional "auto" or number` - - `"input_text"` + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - `OutputText = object { text, type }` + - `"auto"` - A text output from the model. + - `"auto"` - - `text: string` + - `number` - The text output from the model. +- `has_more: boolean` - - `type: "output_text"` +- `object: "list"` - The type of the output text. Always `output_text`. + - `"list"` - - `"output_text"` +### Example - - `InputImage = object { image_url, type, detail }` +```http +curl https://api.openai.com/v1/fine_tuning/jobs \ + -H "Authorization: Bearer $OPENAI_API_KEY" +``` - An image input block used within EvalItem content arrays. +#### Response - - `image_url: string` +```json +{ + "data": [ + { + "id": "id", + "created_at": 0, + "error": { + "code": "code", + "message": "message", + "param": "param" + }, + "fine_tuned_model": "fine_tuned_model", + "finished_at": 0, + "hyperparameters": { + "batch_size": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto" + }, + "model": "model", + "object": "fine_tuning.job", + "organization_id": "organization_id", + "result_files": [ + "file-abc123" + ], + "seed": 0, + "status": "validating_files", + "trained_tokens": 0, + "training_file": "training_file", + "validation_file": "validation_file", + "estimated_finish": 0, + "integrations": [ + { + "type": "wandb", + "wandb": { + "project": "my-wandb-project", + "entity": "entity", + "name": "name", + "tags": [ + "custom-tag" + ] + } + } + ], + "metadata": { + "foo": "string" + }, + "method": { + "type": "supervised", + "dpo": { + "hyperparameters": { + "batch_size": "auto", + "beta": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto" + } + }, + "reinforcement": { + "grader": { + "input": "input", + "name": "name", + "operation": "eq", + "reference": "reference", + "type": "string_check" + }, + "hyperparameters": { + "batch_size": "auto", + "compute_multiplier": "auto", + "eval_interval": "auto", + "eval_samples": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto", + "reasoning_effort": "default" + } + }, + "supervised": { + "hyperparameters": { + "batch_size": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto" + } + } + } + } + ], + "has_more": true, + "object": "list" +} +``` - The URL of the image input. +### Example - - `type: "input_image"` +```http +curl https://api.openai.com/v1/fine_tuning/jobs?limit=2&metadata[key]=value \ + -H "Authorization: Bearer $OPENAI_API_KEY" +``` - The type of the image input. Always `input_image`. +#### Response - - `"input_image"` +```json +{ + "object": "list", + "data": [ + { + "object": "fine_tuning.job", + "id": "ftjob-abc123", + "model": "gpt-4o-mini-2024-07-18", + "created_at": 1721764800, + "fine_tuned_model": null, + "organization_id": "org-123", + "result_files": [], + "status": "queued", + "validation_file": null, + "training_file": "file-abc123", + "metadata": { + "key": "value" + } + }, + { ... }, + { ... } + ], "has_more": true +} +``` - - `detail: optional string` +## Retrieve fine-tuning job - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. +**get** `/fine_tuning/jobs/{fine_tuning_job_id}` - - `ResponseInputAudio = object { input_audio, type }` +Get info about a fine-tuning job. - An audio input to the model. +[Learn more about fine-tuning](/docs/guides/model-optimization) - - `input_audio: object { data, format }` +### Path Parameters - - `data: string` +- `fine_tuning_job_id: string` - Base64-encoded audio data. +### Returns - - `format: "mp3" or "wav"` +- `FineTuningJob object { id, created_at, error, 16 more }` - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + The `fine_tuning.job` object represents a fine-tuning job that has been created through the API. - - `"mp3"` + - `id: string` - - `"wav"` + The object identifier, which can be referenced in the API endpoints. - - `type: "input_audio"` + - `created_at: number` - The type of the input item. Always `input_audio`. + The Unix timestamp (in seconds) for when the fine-tuning job was created. - - `"input_audio"` + - `error: object { code, message, param }` - - `role: "user" or "assistant" or "system" or "developer"` + For fine-tuning jobs that have `failed`, this will contain more information on the cause of the failure. - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. + - `code: string` - - `"user"` + A machine-readable error code. - - `"assistant"` + - `message: string` - - `"system"` + A human-readable error message. - - `"developer"` + - `param: string` - - `type: optional "message"` + The parameter that was invalid, usually `training_file` or `validation_file`. This field will be null if the failure was not parameter-specific. - The type of the message input. Always `message`. + - `fine_tuned_model: string` - - `"message"` + The name of the fine-tuned model that is being created. The value will be null if the fine-tuning job is still running. - - `model: string` + - `finished_at: number` - The model to use for the evaluation. + The Unix timestamp (in seconds) for when the fine-tuning job was finished. The value will be null if the fine-tuning job is still running. - - `name: string` + - `hyperparameters: object { batch_size, learning_rate_multiplier, n_epochs }` - The name of the grader. + The hyperparameters used for the fine-tuning job. This value will only be returned when running `supervised` jobs. - - `type: "score_model"` + - `batch_size: optional "auto" or number` - The object type, which is always `score_model`. + Number of examples in each batch. A larger batch size means that model parameters + are updated less frequently, but with lower variance. - - `"score_model"` + - `"auto"` - - `range: optional array of number` + - `"auto"` - The range of the score. Defaults to `[0, 1]`. + - `number` - - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` + - `learning_rate_multiplier: optional "auto" or number` - The sampling parameters for the model. + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid + overfitting. - - `max_completions_tokens: optional number` + - `"auto"` - The maximum number of tokens the grader model may generate in its response. + - `"auto"` - - `reasoning_effort: optional ReasoningEffort` + - `number` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. + - `n_epochs: optional "auto" or number` - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + The number of epochs to train the model for. An epoch refers to one full cycle + through the training dataset. - - `"none"` + - `"auto"` - - `"minimal"` + - `"auto"` - - `"low"` + - `number` - - `"medium"` + - `model: string` - - `"high"` + The base model that is being fine-tuned. - - `"xhigh"` + - `object: "fine_tuning.job"` - - `seed: optional number` + The object type, which is always "fine_tuning.job". - A seed value to initialize the randomness, during sampling. + - `"fine_tuning.job"` - - `temperature: optional number` + - `organization_id: string` - A higher temperature increases randomness in the outputs. + The organization that owns the fine-tuning job. - - `top_p: optional number` + - `result_files: array of string` - An alternative to temperature for nucleus sampling; 1.0 includes all tokens. + The compiled results file ID(s) for the fine-tuning job. You can retrieve the results with the [Files API](/docs/api-reference/files/retrieve-contents). - - `LabelModelGrader = object { input, labels, model, 3 more }` + - `seed: number` - A LabelModelGrader object which uses a model to assign labels to each item - in the evaluation. + The seed used for the fine-tuning job. - - `input: array of object { content, role, type }` + - `status: "validating_files" or "queued" or "running" or 3 more` - - `content: string or ResponseInputText or object { text, type } or 3 more` + The current status of the fine-tuning job, which can be either `validating_files`, `queued`, `running`, `succeeded`, `failed`, or `cancelled`. - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. + - `"validating_files"` - - `TextInput = string` + - `"queued"` - A text input to the model. + - `"running"` - - `ResponseInputText = object { text, type }` + - `"succeeded"` - A text input to the model. + - `"failed"` - - `text: string` + - `"cancelled"` - The text input to the model. + - `trained_tokens: number` - - `type: "input_text"` + The total number of billable tokens processed by this fine-tuning job. The value will be null if the fine-tuning job is still running. - The type of the input item. Always `input_text`. + - `training_file: string` - - `"input_text"` + The file ID used for training. You can retrieve the training data with the [Files API](/docs/api-reference/files/retrieve-contents). - - `OutputText = object { text, type }` + - `validation_file: string` - A text output from the model. + The file ID used for validation. You can retrieve the validation results with the [Files API](/docs/api-reference/files/retrieve-contents). - - `text: string` + - `estimated_finish: optional number` - The text output from the model. + The Unix timestamp (in seconds) for when the fine-tuning job is estimated to finish. The value will be null if the fine-tuning job is not running. - - `type: "output_text"` + - `integrations: optional array of FineTuningJobWandbIntegrationObject` - The type of the output text. Always `output_text`. + A list of integrations to enable for this fine-tuning job. - - `"output_text"` + - `type: "wandb"` - - `InputImage = object { image_url, type, detail }` + The type of the integration being enabled for the fine-tuning job - An image input block used within EvalItem content arrays. + - `"wandb"` - - `image_url: string` + - `wandb: FineTuningJobWandbIntegration` - The URL of the image input. + The settings for your integration with Weights and Biases. This payload specifies the project that + metrics will be sent to. Optionally, you can set an explicit display name for your run, add tags + to your run, and set a default entity (team, username, etc) to be associated with your run. - - `type: "input_image"` + - `project: string` - The type of the image input. Always `input_image`. + The name of the project that the new run will be created under. - - `"input_image"` + - `entity: optional string` - - `detail: optional string` + The entity to use for the run. This allows you to set the team or username of the WandB user that you would + like associated with the run. If not set, the default entity for the registered WandB API key is used. - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + - `name: optional string` - - `ResponseInputAudio = object { input_audio, type }` + A display name to set for the run. If not set, we will use the Job ID as the name. - An audio input to the model. + - `tags: optional array of string` - - `input_audio: object { data, format }` + A list of tags to be attached to the newly created run. These tags are passed through directly to WandB. Some + default tags are generated by OpenAI: "openai/finetune", "openai/{base-model}", "openai/{ftjob-abcdef}". - - `data: string` + - `metadata: optional Metadata` - Base64-encoded audio data. + Set of 16 key-value pairs that can be attached to an object. This can be + useful for storing additional information about the object in a structured + format, and querying for objects via API or the dashboard. - - `format: "mp3" or "wav"` + Keys are strings with a maximum length of 64 characters. Values are strings + with a maximum length of 512 characters. - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + - `method: optional object { type, dpo, reinforcement, supervised }` - - `"mp3"` + The method used for fine-tuning. - - `"wav"` + - `type: "supervised" or "dpo" or "reinforcement"` - - `type: "input_audio"` + The type of method. Is either `supervised`, `dpo`, or `reinforcement`. - The type of the input item. Always `input_audio`. + - `"supervised"` - - `"input_audio"` + - `"dpo"` - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` + - `"reinforcement"` - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. + - `dpo: optional DpoMethod` - - `TextInput = string` + Configuration for the DPO fine-tuning method. - A text input to the model. + - `hyperparameters: optional DpoHyperparameters` - - `ResponseInputText = object { text, type }` + The hyperparameters used for the DPO fine-tuning job. - A text input to the model. + - `batch_size: optional "auto" or number` - - `text: string` + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - The text input to the model. + - `"auto"` - - `type: "input_text"` + - `"auto"` - The type of the input item. Always `input_text`. + - `number` - - `"input_text"` + - `beta: optional "auto" or number` - - `OutputText = object { text, type }` + The beta value for the DPO method. A higher beta value will increase the weight of the penalty between the policy and reference model. - A text output from the model. + - `"auto"` - - `text: string` + - `"auto"` - The text output from the model. + - `number` - - `type: "output_text"` + - `learning_rate_multiplier: optional "auto" or number` - The type of the output text. Always `output_text`. + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - `"output_text"` + - `"auto"` - - `InputImage = object { image_url, type, detail }` + - `"auto"` - An image input block used within EvalItem content arrays. + - `number` - - `image_url: string` + - `n_epochs: optional "auto" or number` - The URL of the image input. + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - `type: "input_image"` + - `"auto"` - The type of the image input. Always `input_image`. + - `"auto"` - - `"input_image"` + - `number` - - `detail: optional string` + - `reinforcement: optional ReinforcementMethod` - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + Configuration for the reinforcement fine-tuning method. - - `ResponseInputAudio = object { input_audio, type }` + - `grader: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - An audio input to the model. + The grader used for the fine-tuning job. - - `input_audio: object { data, format }` + - `StringCheckGrader object { input, name, operation, 2 more }` - - `data: string` + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - Base64-encoded audio data. + - `input: string` - - `format: "mp3" or "wav"` + The input text. This may include template strings. - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + - `name: string` - - `"mp3"` + The name of the grader. - - `"wav"` + - `operation: "eq" or "ne" or "like" or "ilike"` - - `type: "input_audio"` + The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. - The type of the input item. Always `input_audio`. + - `"eq"` - - `"input_audio"` + - `"ne"` - - `role: "user" or "assistant" or "system" or "developer"` + - `"like"` - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. + - `"ilike"` - - `"user"` + - `reference: string` - - `"assistant"` + The reference text. This may include template strings. - - `"system"` + - `type: "string_check"` - - `"developer"` + The object type, which is always `string_check`. - - `type: optional "message"` + - `"string_check"` - The type of the message input. Always `message`. + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` - - `"message"` + A TextSimilarityGrader object which grades text based on similarity metrics. - - `labels: array of string` + - `evaluation_metric: "cosine" or "fuzzy_match" or "bleu" or 8 more` - The labels to assign to each item in the evaluation. + The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, + `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, + or `rouge_l`. - - `model: string` + - `"cosine"` - The model to use for the evaluation. Must support structured outputs. + - `"fuzzy_match"` - - `name: string` + - `"bleu"` - The name of the grader. + - `"gleu"` - - `passing_labels: array of string` + - `"meteor"` - The labels that indicate a passing result. Must be a subset of labels. + - `"rouge_1"` - - `type: "label_model"` + - `"rouge_2"` - The object type, which is always `label_model`. + - `"rouge_3"` - - `"label_model"` + - `"rouge_4"` - - `name: string` + - `"rouge_5"` - The name of the grader. + - `"rouge_l"` - - `type: "multi"` + - `input: string` - The object type, which is always `multi`. + The text being graded. - - `"multi"` + - `name: string` - - `hyperparameters: optional ReinforcementHyperparameters` + The name of the grader. - The hyperparameters used for the reinforcement fine-tuning job. + - `reference: string` - - `batch_size: optional "auto" or number` + The text being graded against. - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. + - `type: "text_similarity"` - - `"auto"` + The type of grader. - - `"auto"` + - `"text_similarity"` - - `number` + - `PythonGrader object { name, source, type, image_tag }` - - `compute_multiplier: optional "auto" or number` + A PythonGrader object that runs a python script on the input. - Multiplier on amount of compute used for exploring search space during training. + - `name: string` - - `"auto"` + The name of the grader. - - `"auto"` + - `source: string` - - `number` + The source code of the python script. - - `eval_interval: optional "auto" or number` + - `type: "python"` - The number of training steps between evaluation runs. + The object type, which is always `python`. - - `"auto"` + - `"python"` - - `"auto"` + - `image_tag: optional string` - - `number` + The image tag to use for the python script. - - `eval_samples: optional "auto" or number` + - `ScoreModelGrader object { input, model, name, 3 more }` - Number of evaluation samples to generate per training step. + A ScoreModelGrader object that uses a model to assign a score to the input. - - `"auto"` + - `input: array of object { content, role, type }` - - `"auto"` + The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. - - `number` + - `content: string or ResponseInputText or object { text, type } or 3 more` - - `learning_rate_multiplier: optional "auto" or number` + Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. + - `TextInput = string` - - `"auto"` + A text input to the model. - - `"auto"` + - `ResponseInputText object { text, type }` - - `number` + A text input to the model. - - `n_epochs: optional "auto" or number` + - `text: string` - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. + The text input to the model. - - `"auto"` + - `type: "input_text"` - - `"auto"` + The type of the input item. Always `input_text`. - - `number` + - `"input_text"` - - `reasoning_effort: optional "default" or "low" or "medium" or "high"` + - `OutputText object { text, type }` - Level of reasoning effort. + A text output from the model. - - `"default"` + - `text: string` - - `"low"` + The text output from the model. - - `"medium"` + - `type: "output_text"` - - `"high"` + The type of the output text. Always `output_text`. - - `supervised: optional SupervisedMethod` + - `"output_text"` - Configuration for the supervised fine-tuning method. + - `InputImage object { image_url, type, detail }` - - `hyperparameters: optional SupervisedHyperparameters` + An image input block used within EvalItem content arrays. - The hyperparameters used for the fine-tuning job. + - `image_url: string` - - `batch_size: optional "auto" or number` + The URL of the image input. - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. + - `type: "input_image"` - - `"auto"` + The type of the image input. Always `input_image`. - - `"auto"` + - `"input_image"` - - `number` + - `detail: optional string` - - `learning_rate_multiplier: optional "auto" or number` + The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. + - `ResponseInputAudio object { input_audio, type }` - - `"auto"` + An audio input to the model. - - `"auto"` + - `input_audio: object { data, format }` - - `number` + - `data: string` - - `n_epochs: optional "auto" or number` + Base64-encoded audio data. - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. + - `format: "mp3" or "wav"` - - `"auto"` + The format of the audio data. Currently supported formats are `mp3` and + `wav`. - - `"auto"` + - `"mp3"` - - `number` + - `"wav"` -### Example + - `type: "input_audio"` -```http -curl https://api.openai.com/v1/fine_tuning/jobs/$FINE_TUNING_JOB_ID/pause \ - -X POST \ - -H "Authorization: Bearer $OPENAI_API_KEY" -``` + The type of the input item. Always `input_audio`. -#### Response + - `"input_audio"` -```json -{ - "id": "id", - "created_at": 0, - "error": { - "code": "code", - "message": "message", - "param": "param" - }, - "fine_tuned_model": "fine_tuned_model", - "finished_at": 0, - "hyperparameters": { - "batch_size": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto" - }, - "model": "model", - "object": "fine_tuning.job", - "organization_id": "organization_id", - "result_files": [ - "file-abc123" - ], - "seed": 0, - "status": "validating_files", - "trained_tokens": 0, - "training_file": "training_file", - "validation_file": "validation_file", - "estimated_finish": 0, - "integrations": [ - { - "type": "wandb", - "wandb": { - "project": "my-wandb-project", - "entity": "entity", - "name": "name", - "tags": [ - "custom-tag" - ] - } - } - ], - "metadata": { - "foo": "string" - }, - "method": { - "type": "supervised", - "dpo": { - "hyperparameters": { - "batch_size": "auto", - "beta": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto" - } - }, - "reinforcement": { - "grader": { - "input": "input", - "name": "name", - "operation": "eq", - "reference": "reference", - "type": "string_check" - }, - "hyperparameters": { - "batch_size": "auto", - "compute_multiplier": "auto", - "eval_interval": "auto", - "eval_samples": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto", - "reasoning_effort": "default" - } - }, - "supervised": { - "hyperparameters": { - "batch_size": "auto", - "learning_rate_multiplier": "auto", - "n_epochs": "auto" - } - } - } -} -``` + - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` -### Example + A list of inputs, each of which may be either an input text, output text, input + image, or input audio object. -```http -curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/pause \ - -H "Authorization: Bearer $OPENAI_API_KEY" -``` + - `TextInput = string` -#### Response + A text input to the model. -```json -{ - "object": "fine_tuning.job", - "id": "ftjob-abc123", - "model": "gpt-4o-mini-2024-07-18", - "created_at": 1721764800, - "fine_tuned_model": null, - "organization_id": "org-123", - "result_files": [], - "status": "paused", - "validation_file": "file-abc123", - "training_file": "file-abc123" -} -``` + - `ResponseInputText object { text, type }` -## Resume fine-tuning + A text input to the model. -**post** `/fine_tuning/jobs/{fine_tuning_job_id}/resume` + - `OutputText object { text, type }` -Resume a fine-tune job. + A text output from the model. -### Path Parameters + - `text: string` -- `fine_tuning_job_id: string` + The text output from the model. -### Returns + - `type: "output_text"` -- `FineTuningJob = object { id, created_at, error, 16 more }` + The type of the output text. Always `output_text`. - The `fine_tuning.job` object represents a fine-tuning job that has been created through the API. + - `"output_text"` - - `id: string` + - `InputImage object { image_url, type, detail }` - The object identifier, which can be referenced in the API endpoints. + An image input block used within EvalItem content arrays. - - `created_at: number` + - `image_url: string` - The Unix timestamp (in seconds) for when the fine-tuning job was created. + The URL of the image input. - - `error: object { code, message, param }` + - `type: "input_image"` - For fine-tuning jobs that have `failed`, this will contain more information on the cause of the failure. + The type of the image input. Always `input_image`. - - `code: string` + - `"input_image"` - A machine-readable error code. + - `detail: optional string` - - `message: string` + The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - A human-readable error message. + - `ResponseInputAudio object { input_audio, type }` - - `param: string` + An audio input to the model. - The parameter that was invalid, usually `training_file` or `validation_file`. This field will be null if the failure was not parameter-specific. + - `role: "user" or "assistant" or "system" or "developer"` - - `fine_tuned_model: string` + The role of the message input. One of `user`, `assistant`, `system`, or + `developer`. - The name of the fine-tuned model that is being created. The value will be null if the fine-tuning job is still running. + - `"user"` - - `finished_at: number` + - `"assistant"` - The Unix timestamp (in seconds) for when the fine-tuning job was finished. The value will be null if the fine-tuning job is still running. + - `"system"` - - `hyperparameters: object { batch_size, learning_rate_multiplier, n_epochs }` + - `"developer"` - The hyperparameters used for the fine-tuning job. This value will only be returned when running `supervised` jobs. + - `type: optional "message"` - - `batch_size: optional "auto" or number` + The type of the message input. Always `message`. - Number of examples in each batch. A larger batch size means that model parameters - are updated less frequently, but with lower variance. + - `"message"` - - `"auto"` + - `model: string` - - `"auto"` + The model to use for the evaluation. - - `number` + - `name: string` - - `learning_rate_multiplier: optional "auto" or number` + The name of the grader. - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid - overfitting. + - `type: "score_model"` - - `"auto"` + The object type, which is always `score_model`. - - `"auto"` + - `"score_model"` - - `number` + - `range: optional array of number` - - `n_epochs: optional "auto" or number` + The range of the score. Defaults to `[0, 1]`. - The number of epochs to train the model for. An epoch refers to one full cycle - through the training dataset. + - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` - - `"auto"` + The sampling parameters for the model. - - `"auto"` + - `max_completions_tokens: optional number` - - `number` + The maximum number of tokens the grader model may generate in its response. - - `model: string` + - `reasoning_effort: optional ReasoningEffort` - The base model that is being fine-tuned. + Constrains effort on reasoning for + [reasoning models](https://platform.openai.com/docs/guides/reasoning). + Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing + reasoning effort can result in faster responses and fewer tokens used + on reasoning in a response. - - `object: "fine_tuning.job"` + - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. + - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. + - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. + - `xhigh` is supported for all models after `gpt-5.1-codex-max`. - The object type, which is always "fine_tuning.job". + - `"none"` - - `"fine_tuning.job"` + - `"minimal"` - - `organization_id: string` + - `"low"` - The organization that owns the fine-tuning job. + - `"medium"` - - `result_files: array of string` + - `"high"` - The compiled results file ID(s) for the fine-tuning job. You can retrieve the results with the [Files API](/docs/api-reference/files/retrieve-contents). + - `"xhigh"` - - `seed: number` + - `seed: optional number` - The seed used for the fine-tuning job. + A seed value to initialize the randomness, during sampling. - - `status: "validating_files" or "queued" or "running" or 3 more` + - `temperature: optional number` - The current status of the fine-tuning job, which can be either `validating_files`, `queued`, `running`, `succeeded`, `failed`, or `cancelled`. + A higher temperature increases randomness in the outputs. - - `"validating_files"` + - `top_p: optional number` - - `"queued"` + An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `"running"` + - `MultiGrader object { calculate_output, graders, name, type }` - - `"succeeded"` + A MultiGrader object combines the output of multiple graders to produce a single score. - - `"failed"` + - `calculate_output: string` - - `"cancelled"` + A formula to calculate the output based on grader results. - - `trained_tokens: number` + - `graders: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - The total number of billable tokens processed by this fine-tuning job. The value will be null if the fine-tuning job is still running. + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - `training_file: string` + - `StringCheckGrader object { input, name, operation, 2 more }` - The file ID used for training. You can retrieve the training data with the [Files API](/docs/api-reference/files/retrieve-contents). + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - `validation_file: string` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` - The file ID used for validation. You can retrieve the validation results with the [Files API](/docs/api-reference/files/retrieve-contents). + A TextSimilarityGrader object which grades text based on similarity metrics. - - `estimated_finish: optional number` + - `PythonGrader object { name, source, type, image_tag }` - The Unix timestamp (in seconds) for when the fine-tuning job is estimated to finish. The value will be null if the fine-tuning job is not running. + A PythonGrader object that runs a python script on the input. - - `integrations: optional array of FineTuningJobWandbIntegrationObject` + - `ScoreModelGrader object { input, model, name, 3 more }` - A list of integrations to enable for this fine-tuning job. + A ScoreModelGrader object that uses a model to assign a score to the input. - - `type: "wandb"` + - `LabelModelGrader object { input, labels, model, 3 more }` - The type of the integration being enabled for the fine-tuning job + A LabelModelGrader object which uses a model to assign labels to each item + in the evaluation. - - `"wandb"` + - `input: array of object { content, role, type }` - - `wandb: FineTuningJobWandbIntegration` + - `content: string or ResponseInputText or object { text, type } or 3 more` - The settings for your integration with Weights and Biases. This payload specifies the project that - metrics will be sent to. Optionally, you can set an explicit display name for your run, add tags - to your run, and set a default entity (team, username, etc) to be associated with your run. + Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - - `project: string` + - `TextInput = string` + + A text input to the model. + + - `ResponseInputText object { text, type }` + + A text input to the model. + + - `OutputText object { text, type }` + + A text output from the model. + + - `text: string` + + The text output from the model. + + - `type: "output_text"` + + The type of the output text. Always `output_text`. + + - `"output_text"` + + - `InputImage object { image_url, type, detail }` + + An image input block used within EvalItem content arrays. + + - `image_url: string` + + The URL of the image input. + + - `type: "input_image"` + + The type of the image input. Always `input_image`. + + - `"input_image"` + + - `detail: optional string` + + The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + + - `ResponseInputAudio object { input_audio, type }` + + An audio input to the model. + + - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` + + A list of inputs, each of which may be either an input text, output text, input + image, or input audio object. + + - `role: "user" or "assistant" or "system" or "developer"` + + The role of the message input. One of `user`, `assistant`, `system`, or + `developer`. + + - `"user"` + + - `"assistant"` + + - `"system"` + + - `"developer"` + + - `type: optional "message"` + + The type of the message input. Always `message`. + + - `"message"` + + - `labels: array of string` + + The labels to assign to each item in the evaluation. + + - `model: string` + + The model to use for the evaluation. Must support structured outputs. + + - `name: string` + + The name of the grader. + + - `passing_labels: array of string` - The name of the project that the new run will be created under. + The labels that indicate a passing result. Must be a subset of labels. - - `entity: optional string` + - `type: "label_model"` - The entity to use for the run. This allows you to set the team or username of the WandB user that you would - like associated with the run. If not set, the default entity for the registered WandB API key is used. + The object type, which is always `label_model`. - - `name: optional string` + - `"label_model"` - A display name to set for the run. If not set, we will use the Job ID as the name. + - `name: string` - - `tags: optional array of string` + The name of the grader. - A list of tags to be attached to the newly created run. These tags are passed through directly to WandB. Some - default tags are generated by OpenAI: "openai/finetune", "openai/{base-model}", "openai/{ftjob-abcdef}". + - `type: "multi"` - - `metadata: optional Metadata` + The object type, which is always `multi`. - Set of 16 key-value pairs that can be attached to an object. This can be - useful for storing additional information about the object in a structured - format, and querying for objects via API or the dashboard. + - `"multi"` - Keys are strings with a maximum length of 64 characters. Values are strings - with a maximum length of 512 characters. + - `hyperparameters: optional ReinforcementHyperparameters` - - `method: optional object { type, dpo, reinforcement, supervised }` + The hyperparameters used for the reinforcement fine-tuning job. - The method used for fine-tuning. + - `batch_size: optional "auto" or number` - - `type: "supervised" or "dpo" or "reinforcement"` + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - The type of method. Is either `supervised`, `dpo`, or `reinforcement`. + - `"auto"` - - `"supervised"` + - `"auto"` - - `"dpo"` + - `number` - - `"reinforcement"` + - `compute_multiplier: optional "auto" or number` - - `dpo: optional DpoMethod` + Multiplier on amount of compute used for exploring search space during training. - Configuration for the DPO fine-tuning method. + - `"auto"` - - `hyperparameters: optional DpoHyperparameters` + - `"auto"` - The hyperparameters used for the DPO fine-tuning job. + - `number` - - `batch_size: optional "auto" or number` + - `eval_interval: optional "auto" or number` - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. + The number of training steps between evaluation runs. - `"auto"` @@ -10385,9 +4596,9 @@ Resume a fine-tune job. - `number` - - `beta: optional "auto" or number` + - `eval_samples: optional "auto" or number` - The beta value for the DPO method. A higher beta value will increase the weight of the penalty between the policy and reference model. + Number of evaluation samples to generate per training step. - `"auto"` @@ -10415,382 +4626,580 @@ Resume a fine-tune job. - `number` - - `reinforcement: optional ReinforcementMethod` + - `reasoning_effort: optional "default" or "low" or "medium" or "high"` - Configuration for the reinforcement fine-tuning method. + Level of reasoning effort. - - `grader: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` + - `"default"` - The grader used for the fine-tuning job. + - `"low"` - - `StringCheckGrader = object { input, name, operation, 2 more }` + - `"medium"` - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. + - `"high"` - - `input: string` + - `supervised: optional SupervisedMethod` - The input text. This may include template strings. + Configuration for the supervised fine-tuning method. - - `name: string` + - `hyperparameters: optional SupervisedHyperparameters` - The name of the grader. + The hyperparameters used for the fine-tuning job. - - `operation: "eq" or "ne" or "like" or "ilike"` + - `batch_size: optional "auto" or number` - The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - `"eq"` + - `"auto"` - - `"ne"` + - `"auto"` - - `"like"` + - `number` - - `"ilike"` + - `learning_rate_multiplier: optional "auto" or number` - - `reference: string` + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - The reference text. This may include template strings. + - `"auto"` - - `type: "string_check"` + - `"auto"` - The object type, which is always `string_check`. + - `number` - - `"string_check"` + - `n_epochs: optional "auto" or number` - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - A TextSimilarityGrader object which grades text based on similarity metrics. + - `"auto"` - - `evaluation_metric: "cosine" or "fuzzy_match" or "bleu" or 8 more` + - `"auto"` + + - `number` + +### Example + +```http +curl https://api.openai.com/v1/fine_tuning/jobs/$FINE_TUNING_JOB_ID \ + -H "Authorization: Bearer $OPENAI_API_KEY" +``` + +#### Response + +```json +{ + "id": "id", + "created_at": 0, + "error": { + "code": "code", + "message": "message", + "param": "param" + }, + "fine_tuned_model": "fine_tuned_model", + "finished_at": 0, + "hyperparameters": { + "batch_size": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto" + }, + "model": "model", + "object": "fine_tuning.job", + "organization_id": "organization_id", + "result_files": [ + "file-abc123" + ], + "seed": 0, + "status": "validating_files", + "trained_tokens": 0, + "training_file": "training_file", + "validation_file": "validation_file", + "estimated_finish": 0, + "integrations": [ + { + "type": "wandb", + "wandb": { + "project": "my-wandb-project", + "entity": "entity", + "name": "name", + "tags": [ + "custom-tag" + ] + } + } + ], + "metadata": { + "foo": "string" + }, + "method": { + "type": "supervised", + "dpo": { + "hyperparameters": { + "batch_size": "auto", + "beta": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto" + } + }, + "reinforcement": { + "grader": { + "input": "input", + "name": "name", + "operation": "eq", + "reference": "reference", + "type": "string_check" + }, + "hyperparameters": { + "batch_size": "auto", + "compute_multiplier": "auto", + "eval_interval": "auto", + "eval_samples": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto", + "reasoning_effort": "default" + } + }, + "supervised": { + "hyperparameters": { + "batch_size": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto" + } + } + } +} +``` + +### Example + +```http +curl https://api.openai.com/v1/fine_tuning/jobs/ft-AF1WoRqd3aJAHsqc9NY7iL8F \ + -H "Authorization: Bearer $OPENAI_API_KEY" +``` + +#### Response + +```json +{ + "object": "fine_tuning.job", + "id": "ftjob-abc123", + "model": "davinci-002", + "created_at": 1692661014, + "finished_at": 1692661190, + "fine_tuned_model": "ft:davinci-002:my-org:custom_suffix:7q8mpxmy", + "organization_id": "org-123", + "result_files": [ + "file-abc123" + ], + "status": "succeeded", + "validation_file": null, + "training_file": "file-abc123", + "hyperparameters": { + "n_epochs": 4, + "batch_size": 1, + "learning_rate_multiplier": 1.0 + }, + "trained_tokens": 5768, + "integrations": [], + "seed": 0, + "estimated_finish": 0, + "method": { + "type": "supervised", + "supervised": { + "hyperparameters": { + "n_epochs": 4, + "batch_size": 1, + "learning_rate_multiplier": 1.0 + } + } + } +} +``` + +## List fine-tuning events + +**get** `/fine_tuning/jobs/{fine_tuning_job_id}/events` + +Get status updates for a fine-tuning job. + +### Path Parameters + +- `fine_tuning_job_id: string` + +### Query Parameters + +- `after: optional string` - The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, - `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, - or `rouge_l`. + Identifier for the last event from the previous pagination request. - - `"cosine"` +- `limit: optional number` - - `"fuzzy_match"` + Number of events to retrieve. - - `"bleu"` +### Returns - - `"gleu"` +- `data: array of FineTuningJobEvent` - - `"meteor"` + - `id: string` - - `"rouge_1"` + The object identifier. - - `"rouge_2"` + - `created_at: number` - - `"rouge_3"` + The Unix timestamp (in seconds) for when the fine-tuning job was created. - - `"rouge_4"` + - `level: "info" or "warn" or "error"` - - `"rouge_5"` + The log level of the event. - - `"rouge_l"` + - `"info"` - - `input: string` + - `"warn"` - The text being graded. + - `"error"` - - `name: string` + - `message: string` - The name of the grader. + The message of the event. - - `reference: string` + - `object: "fine_tuning.job.event"` - The text being graded against. + The object type, which is always "fine_tuning.job.event". - - `type: "text_similarity"` + - `"fine_tuning.job.event"` - The type of grader. + - `data: optional unknown` - - `"text_similarity"` + The data associated with the event. - - `PythonGrader = object { name, source, type, image_tag }` + - `type: optional "message" or "metrics"` - A PythonGrader object that runs a python script on the input. + The type of event. - - `name: string` + - `"message"` - The name of the grader. + - `"metrics"` - - `source: string` +- `has_more: boolean` - The source code of the python script. +- `object: "list"` - - `type: "python"` + - `"list"` - The object type, which is always `python`. +### Example - - `"python"` +```http +curl https://api.openai.com/v1/fine_tuning/jobs/$FINE_TUNING_JOB_ID/events \ + -H "Authorization: Bearer $OPENAI_API_KEY" +``` - - `image_tag: optional string` +#### Response - The image tag to use for the python script. +```json +{ + "data": [ + { + "id": "id", + "created_at": 0, + "level": "info", + "message": "message", + "object": "fine_tuning.job.event", + "data": {}, + "type": "message" + } + ], + "has_more": true, + "object": "list" +} +``` - - `ScoreModelGrader = object { input, model, name, 3 more }` +### Example - A ScoreModelGrader object that uses a model to assign a score to the input. +```http +curl https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/events \ + -H "Authorization: Bearer $OPENAI_API_KEY" +``` - - `input: array of object { content, role, type }` +#### Response - The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. +```json +{ + "object": "list", + "data": [ + { + "object": "fine_tuning.job.event", + "id": "ft-event-ddTJfwuMVpfLXseO0Am0Gqjm", + "created_at": 1721764800, + "level": "info", + "message": "Fine tuning job successfully completed", + "data": null, + "type": "message" + }, + { + "object": "fine_tuning.job.event", + "id": "ft-event-tyiGuB72evQncpH87xe505Sv", + "created_at": 1721764800, + "level": "info", + "message": "New fine-tuned model created: ft:gpt-4o-mini:openai::7p4lURel", + "data": null, + "type": "message" + } + ], + "has_more": true +} +``` - - `content: string or ResponseInputText or object { text, type } or 3 more` +## Cancel fine-tuning - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. +**post** `/fine_tuning/jobs/{fine_tuning_job_id}/cancel` - - `TextInput = string` +Immediately cancel a fine-tune job. - A text input to the model. +### Path Parameters - - `ResponseInputText = object { text, type }` +- `fine_tuning_job_id: string` - A text input to the model. +### Returns - - `text: string` +- `FineTuningJob object { id, created_at, error, 16 more }` - The text input to the model. + The `fine_tuning.job` object represents a fine-tuning job that has been created through the API. - - `type: "input_text"` + - `id: string` - The type of the input item. Always `input_text`. + The object identifier, which can be referenced in the API endpoints. - - `"input_text"` + - `created_at: number` - - `OutputText = object { text, type }` + The Unix timestamp (in seconds) for when the fine-tuning job was created. - A text output from the model. + - `error: object { code, message, param }` - - `text: string` + For fine-tuning jobs that have `failed`, this will contain more information on the cause of the failure. - The text output from the model. + - `code: string` - - `type: "output_text"` + A machine-readable error code. - The type of the output text. Always `output_text`. + - `message: string` - - `"output_text"` + A human-readable error message. - - `InputImage = object { image_url, type, detail }` + - `param: string` - An image input block used within EvalItem content arrays. + The parameter that was invalid, usually `training_file` or `validation_file`. This field will be null if the failure was not parameter-specific. - - `image_url: string` + - `fine_tuned_model: string` - The URL of the image input. + The name of the fine-tuned model that is being created. The value will be null if the fine-tuning job is still running. - - `type: "input_image"` + - `finished_at: number` - The type of the image input. Always `input_image`. + The Unix timestamp (in seconds) for when the fine-tuning job was finished. The value will be null if the fine-tuning job is still running. - - `"input_image"` + - `hyperparameters: object { batch_size, learning_rate_multiplier, n_epochs }` - - `detail: optional string` + The hyperparameters used for the fine-tuning job. This value will only be returned when running `supervised` jobs. - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + - `batch_size: optional "auto" or number` - - `ResponseInputAudio = object { input_audio, type }` + Number of examples in each batch. A larger batch size means that model parameters + are updated less frequently, but with lower variance. - An audio input to the model. + - `"auto"` - - `input_audio: object { data, format }` + - `"auto"` - - `data: string` + - `number` - Base64-encoded audio data. + - `learning_rate_multiplier: optional "auto" or number` - - `format: "mp3" or "wav"` + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid + overfitting. - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + - `"auto"` - - `"mp3"` + - `"auto"` - - `"wav"` + - `number` - - `type: "input_audio"` + - `n_epochs: optional "auto" or number` - The type of the input item. Always `input_audio`. + The number of epochs to train the model for. An epoch refers to one full cycle + through the training dataset. - - `"input_audio"` + - `"auto"` - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` + - `"auto"` - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. + - `number` - - `TextInput = string` + - `model: string` - A text input to the model. + The base model that is being fine-tuned. - - `ResponseInputText = object { text, type }` + - `object: "fine_tuning.job"` - A text input to the model. + The object type, which is always "fine_tuning.job". - - `text: string` + - `"fine_tuning.job"` - The text input to the model. + - `organization_id: string` - - `type: "input_text"` + The organization that owns the fine-tuning job. - The type of the input item. Always `input_text`. + - `result_files: array of string` - - `"input_text"` + The compiled results file ID(s) for the fine-tuning job. You can retrieve the results with the [Files API](/docs/api-reference/files/retrieve-contents). - - `OutputText = object { text, type }` + - `seed: number` - A text output from the model. + The seed used for the fine-tuning job. - - `text: string` + - `status: "validating_files" or "queued" or "running" or 3 more` - The text output from the model. + The current status of the fine-tuning job, which can be either `validating_files`, `queued`, `running`, `succeeded`, `failed`, or `cancelled`. - - `type: "output_text"` + - `"validating_files"` - The type of the output text. Always `output_text`. + - `"queued"` - - `"output_text"` + - `"running"` - - `InputImage = object { image_url, type, detail }` + - `"succeeded"` - An image input block used within EvalItem content arrays. + - `"failed"` - - `image_url: string` + - `"cancelled"` - The URL of the image input. + - `trained_tokens: number` - - `type: "input_image"` + The total number of billable tokens processed by this fine-tuning job. The value will be null if the fine-tuning job is still running. - The type of the image input. Always `input_image`. + - `training_file: string` - - `"input_image"` + The file ID used for training. You can retrieve the training data with the [Files API](/docs/api-reference/files/retrieve-contents). - - `detail: optional string` + - `validation_file: string` - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + The file ID used for validation. You can retrieve the validation results with the [Files API](/docs/api-reference/files/retrieve-contents). - - `ResponseInputAudio = object { input_audio, type }` + - `estimated_finish: optional number` - An audio input to the model. + The Unix timestamp (in seconds) for when the fine-tuning job is estimated to finish. The value will be null if the fine-tuning job is not running. - - `input_audio: object { data, format }` + - `integrations: optional array of FineTuningJobWandbIntegrationObject` - - `data: string` + A list of integrations to enable for this fine-tuning job. - Base64-encoded audio data. + - `type: "wandb"` - - `format: "mp3" or "wav"` + The type of the integration being enabled for the fine-tuning job - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + - `"wandb"` - - `"mp3"` + - `wandb: FineTuningJobWandbIntegration` - - `"wav"` + The settings for your integration with Weights and Biases. This payload specifies the project that + metrics will be sent to. Optionally, you can set an explicit display name for your run, add tags + to your run, and set a default entity (team, username, etc) to be associated with your run. - - `type: "input_audio"` + - `project: string` - The type of the input item. Always `input_audio`. + The name of the project that the new run will be created under. - - `"input_audio"` + - `entity: optional string` - - `role: "user" or "assistant" or "system" or "developer"` + The entity to use for the run. This allows you to set the team or username of the WandB user that you would + like associated with the run. If not set, the default entity for the registered WandB API key is used. - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. + - `name: optional string` - - `"user"` + A display name to set for the run. If not set, we will use the Job ID as the name. - - `"assistant"` + - `tags: optional array of string` - - `"system"` + A list of tags to be attached to the newly created run. These tags are passed through directly to WandB. Some + default tags are generated by OpenAI: "openai/finetune", "openai/{base-model}", "openai/{ftjob-abcdef}". - - `"developer"` + - `metadata: optional Metadata` - - `type: optional "message"` + Set of 16 key-value pairs that can be attached to an object. This can be + useful for storing additional information about the object in a structured + format, and querying for objects via API or the dashboard. - The type of the message input. Always `message`. + Keys are strings with a maximum length of 64 characters. Values are strings + with a maximum length of 512 characters. - - `"message"` + - `method: optional object { type, dpo, reinforcement, supervised }` - - `model: string` + The method used for fine-tuning. - The model to use for the evaluation. + - `type: "supervised" or "dpo" or "reinforcement"` - - `name: string` + The type of method. Is either `supervised`, `dpo`, or `reinforcement`. - The name of the grader. + - `"supervised"` - - `type: "score_model"` + - `"dpo"` - The object type, which is always `score_model`. + - `"reinforcement"` - - `"score_model"` + - `dpo: optional DpoMethod` - - `range: optional array of number` + Configuration for the DPO fine-tuning method. - The range of the score. Defaults to `[0, 1]`. + - `hyperparameters: optional DpoHyperparameters` - - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` + The hyperparameters used for the DPO fine-tuning job. - The sampling parameters for the model. + - `batch_size: optional "auto" or number` - - `max_completions_tokens: optional number` + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - The maximum number of tokens the grader model may generate in its response. + - `"auto"` - - `reasoning_effort: optional ReasoningEffort` + - `"auto"` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. + - `number` - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + - `beta: optional "auto" or number` - - `"none"` + The beta value for the DPO method. A higher beta value will increase the weight of the penalty between the policy and reference model. - - `"minimal"` + - `"auto"` - - `"low"` + - `"auto"` - - `"medium"` + - `number` - - `"high"` + - `learning_rate_multiplier: optional "auto" or number` - - `"xhigh"` + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - `seed: optional number` + - `"auto"` - A seed value to initialize the randomness, during sampling. + - `"auto"` - - `temperature: optional number` + - `number` - A higher temperature increases randomness in the outputs. + - `n_epochs: optional "auto" or number` - - `top_p: optional number` + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - An alternative to temperature for nucleus sampling; 1.0 includes all tokens. + - `"auto"` - - `MultiGrader = object { calculate_output, graders, name, type }` + - `"auto"` - A MultiGrader object combines the output of multiple graders to produce a single score. + - `number` - - `calculate_output: string` + - `reinforcement: optional ReinforcementMethod` - A formula to calculate the output based on grader results. + Configuration for the reinforcement fine-tuning method. - - `graders: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` + - `grader: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. + The grader used for the fine-tuning job. - - `StringCheckGrader = object { input, name, operation, 2 more }` + - `StringCheckGrader object { input, name, operation, 2 more }` A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. @@ -10824,7 +5233,7 @@ Resume a fine-tune job. - `"string_check"` - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` A TextSimilarityGrader object which grades text based on similarity metrics. @@ -10874,7 +5283,7 @@ Resume a fine-tune job. - `"text_similarity"` - - `PythonGrader = object { name, source, type, image_tag }` + - `PythonGrader object { name, source, type, image_tag }` A PythonGrader object that runs a python script on the input. @@ -10896,7 +5305,7 @@ Resume a fine-tune job. The image tag to use for the python script. - - `ScoreModelGrader = object { input, model, name, 3 more }` + - `ScoreModelGrader object { input, model, name, 3 more }` A ScoreModelGrader object that uses a model to assign a score to the input. @@ -10912,7 +5321,7 @@ Resume a fine-tune job. A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. @@ -10926,7 +5335,7 @@ Resume a fine-tune job. - `"input_text"` - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -10940,7 +5349,7 @@ Resume a fine-tune job. - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -10958,7 +5367,7 @@ Resume a fine-tune job. The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. @@ -10992,21 +5401,11 @@ Resume a fine-tune job. A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -11020,7 +5419,7 @@ Resume a fine-tune job. - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -11038,31 +5437,10 @@ Resume a fine-tune job. The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or @@ -11145,116 +5523,54 @@ Resume a fine-tune job. An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `LabelModelGrader = object { input, labels, model, 3 more }` - - A LabelModelGrader object which uses a model to assign labels to each item - in the evaluation. - - - `input: array of object { content, role, type }` - - - `content: string or ResponseInputText or object { text, type } or 3 more` - - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` + - `MultiGrader object { calculate_output, graders, name, type }` - The type of the image input. Always `input_image`. + A MultiGrader object combines the output of multiple graders to produce a single score. - - `"input_image"` + - `calculate_output: string` - - `detail: optional string` + A formula to calculate the output based on grader results. - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + - `graders: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - - `ResponseInputAudio = object { input_audio, type }` + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - An audio input to the model. + - `StringCheckGrader object { input, name, operation, 2 more }` - - `input_audio: object { data, format }` + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - `data: string` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` - Base64-encoded audio data. + A TextSimilarityGrader object which grades text based on similarity metrics. - - `format: "mp3" or "wav"` + - `PythonGrader object { name, source, type, image_tag }` - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + A PythonGrader object that runs a python script on the input. - - `"mp3"` + - `ScoreModelGrader object { input, model, name, 3 more }` - - `"wav"` + A ScoreModelGrader object that uses a model to assign a score to the input. - - `type: "input_audio"` + - `LabelModelGrader object { input, labels, model, 3 more }` - The type of the input item. Always `input_audio`. + A LabelModelGrader object which uses a model to assign labels to each item + in the evaluation. - - `"input_audio"` + - `input: array of object { content, role, type }` - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` + - `content: string or ResponseInputText or object { text, type } or 3 more` - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. + Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - `TextInput = string` A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -11268,7 +5584,7 @@ Resume a fine-tune job. - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -11282,34 +5598,18 @@ Resume a fine-tune job. - `"input_image"` - - `detail: optional string` - - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - - `ResponseInputAudio = object { input_audio, type }` - - An audio input to the model. - - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` + - `detail: optional string` - - `"wav"` + The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `type: "input_audio"` + - `ResponseInputAudio object { input_audio, type }` - The type of the input item. Always `input_audio`. + An audio input to the model. - - `"input_audio"` + - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` + + A list of inputs, each of which may be either an input text, output text, input + image, or input audio object. - `role: "user" or "assistant" or "system" or "developer"` @@ -11479,7 +5779,7 @@ Resume a fine-tune job. ### Example ```http -curl https://api.openai.com/v1/fine_tuning/jobs/$FINE_TUNING_JOB_ID/resume \ +curl https://api.openai.com/v1/fine_tuning/jobs/$FINE_TUNING_JOB_ID/cancel \ -X POST \ -H "Authorization: Bearer $OPENAI_API_KEY" ``` @@ -11572,7 +5872,7 @@ curl https://api.openai.com/v1/fine_tuning/jobs/$FINE_TUNING_JOB_ID/resume \ ### Example ```http -curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/resume \ +curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/cancel \ -H "Authorization: Bearer $OPENAI_API_KEY" ``` @@ -11587,17 +5887,25 @@ curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/resume \ "fine_tuned_model": null, "organization_id": "org-123", "result_files": [], - "status": "queued", + "status": "cancelled", "validation_file": "file-abc123", "training_file": "file-abc123" } ``` -## Domain Types +## Pause fine-tuning -### Fine Tuning Job +**post** `/fine_tuning/jobs/{fine_tuning_job_id}/pause` + +Pause a fine-tune job. -- `FineTuningJob = object { id, created_at, error, 16 more }` +### Path Parameters + +- `fine_tuning_job_id: string` + +### Returns + +- `FineTuningJob object { id, created_at, error, 16 more }` The `fine_tuning.job` object represents a fine-tuning job that has been created through the API. @@ -11815,141 +6123,386 @@ curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/resume \ - `"auto"` - - `"auto"` + - `"auto"` + + - `number` + + - `n_epochs: optional "auto" or number` + + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. + + - `"auto"` + + - `"auto"` + + - `number` + + - `reinforcement: optional ReinforcementMethod` + + Configuration for the reinforcement fine-tuning method. + + - `grader: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` + + The grader used for the fine-tuning job. + + - `StringCheckGrader object { input, name, operation, 2 more }` + + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. + + - `input: string` + + The input text. This may include template strings. + + - `name: string` + + The name of the grader. + + - `operation: "eq" or "ne" or "like" or "ilike"` + + The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. + + - `"eq"` + + - `"ne"` + + - `"like"` + + - `"ilike"` + + - `reference: string` + + The reference text. This may include template strings. + + - `type: "string_check"` + + The object type, which is always `string_check`. + + - `"string_check"` + + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` + + A TextSimilarityGrader object which grades text based on similarity metrics. + + - `evaluation_metric: "cosine" or "fuzzy_match" or "bleu" or 8 more` + + The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, + `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, + or `rouge_l`. + + - `"cosine"` + + - `"fuzzy_match"` + + - `"bleu"` + + - `"gleu"` + + - `"meteor"` + + - `"rouge_1"` + + - `"rouge_2"` + + - `"rouge_3"` + + - `"rouge_4"` + + - `"rouge_5"` + + - `"rouge_l"` + + - `input: string` + + The text being graded. + + - `name: string` + + The name of the grader. + + - `reference: string` + + The text being graded against. + + - `type: "text_similarity"` + + The type of grader. + + - `"text_similarity"` + + - `PythonGrader object { name, source, type, image_tag }` + + A PythonGrader object that runs a python script on the input. + + - `name: string` + + The name of the grader. + + - `source: string` + + The source code of the python script. + + - `type: "python"` + + The object type, which is always `python`. + + - `"python"` + + - `image_tag: optional string` + + The image tag to use for the python script. + + - `ScoreModelGrader object { input, model, name, 3 more }` + + A ScoreModelGrader object that uses a model to assign a score to the input. + + - `input: array of object { content, role, type }` + + The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. + + - `content: string or ResponseInputText or object { text, type } or 3 more` + + Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. + + - `TextInput = string` + + A text input to the model. + + - `ResponseInputText object { text, type }` + + A text input to the model. + + - `text: string` + + The text input to the model. + + - `type: "input_text"` + + The type of the input item. Always `input_text`. + + - `"input_text"` + + - `OutputText object { text, type }` + + A text output from the model. + + - `text: string` + + The text output from the model. + + - `type: "output_text"` + + The type of the output text. Always `output_text`. + + - `"output_text"` + + - `InputImage object { image_url, type, detail }` + + An image input block used within EvalItem content arrays. + + - `image_url: string` + + The URL of the image input. + + - `type: "input_image"` + + The type of the image input. Always `input_image`. + + - `"input_image"` + + - `detail: optional string` + + The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + + - `ResponseInputAudio object { input_audio, type }` + + An audio input to the model. + + - `input_audio: object { data, format }` + + - `data: string` + + Base64-encoded audio data. + + - `format: "mp3" or "wav"` + + The format of the audio data. Currently supported formats are `mp3` and + `wav`. + + - `"mp3"` + + - `"wav"` + + - `type: "input_audio"` + + The type of the input item. Always `input_audio`. + + - `"input_audio"` + + - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` + + A list of inputs, each of which may be either an input text, output text, input + image, or input audio object. + + - `TextInput = string` + + A text input to the model. + + - `ResponseInputText object { text, type }` + + A text input to the model. + + - `OutputText object { text, type }` + + A text output from the model. + + - `text: string` + + The text output from the model. + + - `type: "output_text"` - - `number` + The type of the output text. Always `output_text`. - - `n_epochs: optional "auto" or number` + - `"output_text"` - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. + - `InputImage object { image_url, type, detail }` - - `"auto"` + An image input block used within EvalItem content arrays. - - `"auto"` + - `image_url: string` - - `number` + The URL of the image input. - - `reinforcement: optional ReinforcementMethod` + - `type: "input_image"` - Configuration for the reinforcement fine-tuning method. + The type of the image input. Always `input_image`. - - `grader: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` + - `"input_image"` - The grader used for the fine-tuning job. + - `detail: optional string` - - `StringCheckGrader = object { input, name, operation, 2 more }` + The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. + - `ResponseInputAudio object { input_audio, type }` - - `input: string` + An audio input to the model. - The input text. This may include template strings. + - `role: "user" or "assistant" or "system" or "developer"` - - `name: string` + The role of the message input. One of `user`, `assistant`, `system`, or + `developer`. - The name of the grader. + - `"user"` - - `operation: "eq" or "ne" or "like" or "ilike"` + - `"assistant"` - The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. + - `"system"` - - `"eq"` + - `"developer"` - - `"ne"` + - `type: optional "message"` - - `"like"` + The type of the message input. Always `message`. - - `"ilike"` + - `"message"` - - `reference: string` + - `model: string` - The reference text. This may include template strings. + The model to use for the evaluation. - - `type: "string_check"` + - `name: string` - The object type, which is always `string_check`. + The name of the grader. - - `"string_check"` + - `type: "score_model"` - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` + The object type, which is always `score_model`. - A TextSimilarityGrader object which grades text based on similarity metrics. + - `"score_model"` - - `evaluation_metric: "cosine" or "fuzzy_match" or "bleu" or 8 more` + - `range: optional array of number` - The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, - `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, - or `rouge_l`. + The range of the score. Defaults to `[0, 1]`. - - `"cosine"` + - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` - - `"fuzzy_match"` + The sampling parameters for the model. - - `"bleu"` + - `max_completions_tokens: optional number` - - `"gleu"` + The maximum number of tokens the grader model may generate in its response. - - `"meteor"` + - `reasoning_effort: optional ReasoningEffort` - - `"rouge_1"` + Constrains effort on reasoning for + [reasoning models](https://platform.openai.com/docs/guides/reasoning). + Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing + reasoning effort can result in faster responses and fewer tokens used + on reasoning in a response. - - `"rouge_2"` + - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. + - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. + - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. + - `xhigh` is supported for all models after `gpt-5.1-codex-max`. - - `"rouge_3"` + - `"none"` - - `"rouge_4"` + - `"minimal"` - - `"rouge_5"` + - `"low"` - - `"rouge_l"` + - `"medium"` - - `input: string` + - `"high"` - The text being graded. + - `"xhigh"` - - `name: string` + - `seed: optional number` - The name of the grader. + A seed value to initialize the randomness, during sampling. - - `reference: string` + - `temperature: optional number` - The text being graded against. + A higher temperature increases randomness in the outputs. - - `type: "text_similarity"` + - `top_p: optional number` - The type of grader. + An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `"text_similarity"` + - `MultiGrader object { calculate_output, graders, name, type }` - - `PythonGrader = object { name, source, type, image_tag }` + A MultiGrader object combines the output of multiple graders to produce a single score. - A PythonGrader object that runs a python script on the input. + - `calculate_output: string` - - `name: string` + A formula to calculate the output based on grader results. - The name of the grader. + - `graders: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - - `source: string` + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - The source code of the python script. + - `StringCheckGrader object { input, name, operation, 2 more }` - - `type: "python"` + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - The object type, which is always `python`. + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` - - `"python"` + A TextSimilarityGrader object which grades text based on similarity metrics. - - `image_tag: optional string` + - `PythonGrader object { name, source, type, image_tag }` - The image tag to use for the python script. + A PythonGrader object that runs a python script on the input. - - `ScoreModelGrader = object { input, model, name, 3 more }` + - `ScoreModelGrader object { input, model, name, 3 more }` A ScoreModelGrader object that uses a model to assign a score to the input. - - `input: array of object { content, role, type }` + - `LabelModelGrader object { input, labels, model, 3 more }` - The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. + A LabelModelGrader object which uses a model to assign labels to each item + in the evaluation. + + - `input: array of object { content, role, type }` - `content: string or ResponseInputText or object { text, type } or 3 more` @@ -11959,21 +6512,11 @@ curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/resume \ A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -11987,7 +6530,7 @@ curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/resume \ - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -12005,567 +6548,663 @@ curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/resume \ The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. - - `input_audio: object { data, format }` + - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - `data: string` + A list of inputs, each of which may be either an input text, output text, input + image, or input audio object. - Base64-encoded audio data. + - `role: "user" or "assistant" or "system" or "developer"` - - `format: "mp3" or "wav"` + The role of the message input. One of `user`, `assistant`, `system`, or + `developer`. - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + - `"user"` - - `"mp3"` + - `"assistant"` - - `"wav"` + - `"system"` - - `type: "input_audio"` + - `"developer"` - The type of the input item. Always `input_audio`. + - `type: optional "message"` - - `"input_audio"` + The type of the message input. Always `message`. - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` + - `"message"` - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. + - `labels: array of string` - - `TextInput = string` + The labels to assign to each item in the evaluation. - A text input to the model. + - `model: string` - - `ResponseInputText = object { text, type }` + The model to use for the evaluation. Must support structured outputs. - A text input to the model. + - `name: string` - - `text: string` + The name of the grader. - The text input to the model. + - `passing_labels: array of string` - - `type: "input_text"` + The labels that indicate a passing result. Must be a subset of labels. - The type of the input item. Always `input_text`. + - `type: "label_model"` - - `"input_text"` + The object type, which is always `label_model`. - - `OutputText = object { text, type }` + - `"label_model"` - A text output from the model. + - `name: string` - - `text: string` + The name of the grader. - The text output from the model. + - `type: "multi"` - - `type: "output_text"` + The object type, which is always `multi`. - The type of the output text. Always `output_text`. + - `"multi"` - - `"output_text"` + - `hyperparameters: optional ReinforcementHyperparameters` - - `InputImage = object { image_url, type, detail }` + The hyperparameters used for the reinforcement fine-tuning job. - An image input block used within EvalItem content arrays. + - `batch_size: optional "auto" or number` - - `image_url: string` + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - The URL of the image input. + - `"auto"` - - `type: "input_image"` + - `"auto"` - The type of the image input. Always `input_image`. + - `number` - - `"input_image"` + - `compute_multiplier: optional "auto" or number` - - `detail: optional string` + Multiplier on amount of compute used for exploring search space during training. - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + - `"auto"` - - `ResponseInputAudio = object { input_audio, type }` + - `"auto"` - An audio input to the model. + - `number` - - `input_audio: object { data, format }` + - `eval_interval: optional "auto" or number` - - `data: string` + The number of training steps between evaluation runs. - Base64-encoded audio data. + - `"auto"` - - `format: "mp3" or "wav"` + - `"auto"` - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + - `number` - - `"mp3"` + - `eval_samples: optional "auto" or number` - - `"wav"` + Number of evaluation samples to generate per training step. - - `type: "input_audio"` + - `"auto"` - The type of the input item. Always `input_audio`. + - `"auto"` - - `"input_audio"` + - `number` - - `role: "user" or "assistant" or "system" or "developer"` + - `learning_rate_multiplier: optional "auto" or number` - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - `"user"` + - `"auto"` - - `"assistant"` + - `"auto"` - - `"system"` + - `number` + + - `n_epochs: optional "auto" or number` + + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - `"developer"` + - `"auto"` - - `type: optional "message"` + - `"auto"` - The type of the message input. Always `message`. + - `number` - - `"message"` + - `reasoning_effort: optional "default" or "low" or "medium" or "high"` - - `model: string` + Level of reasoning effort. - The model to use for the evaluation. + - `"default"` - - `name: string` + - `"low"` - The name of the grader. + - `"medium"` - - `type: "score_model"` + - `"high"` - The object type, which is always `score_model`. + - `supervised: optional SupervisedMethod` - - `"score_model"` + Configuration for the supervised fine-tuning method. - - `range: optional array of number` + - `hyperparameters: optional SupervisedHyperparameters` - The range of the score. Defaults to `[0, 1]`. + The hyperparameters used for the fine-tuning job. - - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` + - `batch_size: optional "auto" or number` - The sampling parameters for the model. + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - `max_completions_tokens: optional number` + - `"auto"` - The maximum number of tokens the grader model may generate in its response. + - `"auto"` - - `reasoning_effort: optional ReasoningEffort` + - `number` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. + - `learning_rate_multiplier: optional "auto" or number` - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - `"none"` + - `"auto"` - - `"minimal"` + - `"auto"` - - `"low"` + - `number` - - `"medium"` + - `n_epochs: optional "auto" or number` - - `"high"` + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - `"xhigh"` + - `"auto"` - - `seed: optional number` + - `"auto"` - A seed value to initialize the randomness, during sampling. + - `number` - - `temperature: optional number` +### Example - A higher temperature increases randomness in the outputs. +```http +curl https://api.openai.com/v1/fine_tuning/jobs/$FINE_TUNING_JOB_ID/pause \ + -X POST \ + -H "Authorization: Bearer $OPENAI_API_KEY" +``` - - `top_p: optional number` +#### Response - An alternative to temperature for nucleus sampling; 1.0 includes all tokens. +```json +{ + "id": "id", + "created_at": 0, + "error": { + "code": "code", + "message": "message", + "param": "param" + }, + "fine_tuned_model": "fine_tuned_model", + "finished_at": 0, + "hyperparameters": { + "batch_size": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto" + }, + "model": "model", + "object": "fine_tuning.job", + "organization_id": "organization_id", + "result_files": [ + "file-abc123" + ], + "seed": 0, + "status": "validating_files", + "trained_tokens": 0, + "training_file": "training_file", + "validation_file": "validation_file", + "estimated_finish": 0, + "integrations": [ + { + "type": "wandb", + "wandb": { + "project": "my-wandb-project", + "entity": "entity", + "name": "name", + "tags": [ + "custom-tag" + ] + } + } + ], + "metadata": { + "foo": "string" + }, + "method": { + "type": "supervised", + "dpo": { + "hyperparameters": { + "batch_size": "auto", + "beta": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto" + } + }, + "reinforcement": { + "grader": { + "input": "input", + "name": "name", + "operation": "eq", + "reference": "reference", + "type": "string_check" + }, + "hyperparameters": { + "batch_size": "auto", + "compute_multiplier": "auto", + "eval_interval": "auto", + "eval_samples": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto", + "reasoning_effort": "default" + } + }, + "supervised": { + "hyperparameters": { + "batch_size": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto" + } + } + } +} +``` - - `MultiGrader = object { calculate_output, graders, name, type }` +### Example - A MultiGrader object combines the output of multiple graders to produce a single score. +```http +curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/pause \ + -H "Authorization: Bearer $OPENAI_API_KEY" +``` - - `calculate_output: string` +#### Response - A formula to calculate the output based on grader results. +```json +{ + "object": "fine_tuning.job", + "id": "ftjob-abc123", + "model": "gpt-4o-mini-2024-07-18", + "created_at": 1721764800, + "fine_tuned_model": null, + "organization_id": "org-123", + "result_files": [], + "status": "paused", + "validation_file": "file-abc123", + "training_file": "file-abc123" +} +``` - - `graders: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` +## Resume fine-tuning - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. +**post** `/fine_tuning/jobs/{fine_tuning_job_id}/resume` - - `StringCheckGrader = object { input, name, operation, 2 more }` +Resume a fine-tune job. - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. +### Path Parameters - - `input: string` +- `fine_tuning_job_id: string` - The input text. This may include template strings. +### Returns - - `name: string` +- `FineTuningJob object { id, created_at, error, 16 more }` - The name of the grader. + The `fine_tuning.job` object represents a fine-tuning job that has been created through the API. - - `operation: "eq" or "ne" or "like" or "ilike"` + - `id: string` - The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. + The object identifier, which can be referenced in the API endpoints. - - `"eq"` + - `created_at: number` - - `"ne"` + The Unix timestamp (in seconds) for when the fine-tuning job was created. - - `"like"` + - `error: object { code, message, param }` - - `"ilike"` + For fine-tuning jobs that have `failed`, this will contain more information on the cause of the failure. - - `reference: string` + - `code: string` - The reference text. This may include template strings. + A machine-readable error code. - - `type: "string_check"` + - `message: string` - The object type, which is always `string_check`. + A human-readable error message. - - `"string_check"` + - `param: string` - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` + The parameter that was invalid, usually `training_file` or `validation_file`. This field will be null if the failure was not parameter-specific. - A TextSimilarityGrader object which grades text based on similarity metrics. + - `fine_tuned_model: string` - - `evaluation_metric: "cosine" or "fuzzy_match" or "bleu" or 8 more` + The name of the fine-tuned model that is being created. The value will be null if the fine-tuning job is still running. - The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, - `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, - or `rouge_l`. + - `finished_at: number` - - `"cosine"` + The Unix timestamp (in seconds) for when the fine-tuning job was finished. The value will be null if the fine-tuning job is still running. - - `"fuzzy_match"` + - `hyperparameters: object { batch_size, learning_rate_multiplier, n_epochs }` - - `"bleu"` + The hyperparameters used for the fine-tuning job. This value will only be returned when running `supervised` jobs. - - `"gleu"` + - `batch_size: optional "auto" or number` - - `"meteor"` + Number of examples in each batch. A larger batch size means that model parameters + are updated less frequently, but with lower variance. - - `"rouge_1"` + - `"auto"` - - `"rouge_2"` + - `"auto"` - - `"rouge_3"` + - `number` - - `"rouge_4"` + - `learning_rate_multiplier: optional "auto" or number` - - `"rouge_5"` + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid + overfitting. - - `"rouge_l"` + - `"auto"` - - `input: string` + - `"auto"` - The text being graded. + - `number` - - `name: string` + - `n_epochs: optional "auto" or number` - The name of the grader. + The number of epochs to train the model for. An epoch refers to one full cycle + through the training dataset. - - `reference: string` + - `"auto"` - The text being graded against. + - `"auto"` - - `type: "text_similarity"` + - `number` - The type of grader. + - `model: string` - - `"text_similarity"` + The base model that is being fine-tuned. - - `PythonGrader = object { name, source, type, image_tag }` + - `object: "fine_tuning.job"` - A PythonGrader object that runs a python script on the input. + The object type, which is always "fine_tuning.job". - - `name: string` + - `"fine_tuning.job"` - The name of the grader. + - `organization_id: string` - - `source: string` + The organization that owns the fine-tuning job. - The source code of the python script. + - `result_files: array of string` - - `type: "python"` + The compiled results file ID(s) for the fine-tuning job. You can retrieve the results with the [Files API](/docs/api-reference/files/retrieve-contents). - The object type, which is always `python`. + - `seed: number` - - `"python"` + The seed used for the fine-tuning job. - - `image_tag: optional string` + - `status: "validating_files" or "queued" or "running" or 3 more` - The image tag to use for the python script. + The current status of the fine-tuning job, which can be either `validating_files`, `queued`, `running`, `succeeded`, `failed`, or `cancelled`. - - `ScoreModelGrader = object { input, model, name, 3 more }` + - `"validating_files"` - A ScoreModelGrader object that uses a model to assign a score to the input. + - `"queued"` - - `input: array of object { content, role, type }` + - `"running"` - The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. + - `"succeeded"` - - `content: string or ResponseInputText or object { text, type } or 3 more` + - `"failed"` - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. + - `"cancelled"` - - `TextInput = string` + - `trained_tokens: number` - A text input to the model. + The total number of billable tokens processed by this fine-tuning job. The value will be null if the fine-tuning job is still running. - - `ResponseInputText = object { text, type }` + - `training_file: string` - A text input to the model. + The file ID used for training. You can retrieve the training data with the [Files API](/docs/api-reference/files/retrieve-contents). - - `text: string` + - `validation_file: string` - The text input to the model. + The file ID used for validation. You can retrieve the validation results with the [Files API](/docs/api-reference/files/retrieve-contents). - - `type: "input_text"` + - `estimated_finish: optional number` - The type of the input item. Always `input_text`. + The Unix timestamp (in seconds) for when the fine-tuning job is estimated to finish. The value will be null if the fine-tuning job is not running. - - `"input_text"` + - `integrations: optional array of FineTuningJobWandbIntegrationObject` - - `OutputText = object { text, type }` + A list of integrations to enable for this fine-tuning job. - A text output from the model. + - `type: "wandb"` - - `text: string` + The type of the integration being enabled for the fine-tuning job - The text output from the model. + - `"wandb"` - - `type: "output_text"` + - `wandb: FineTuningJobWandbIntegration` - The type of the output text. Always `output_text`. + The settings for your integration with Weights and Biases. This payload specifies the project that + metrics will be sent to. Optionally, you can set an explicit display name for your run, add tags + to your run, and set a default entity (team, username, etc) to be associated with your run. - - `"output_text"` + - `project: string` - - `InputImage = object { image_url, type, detail }` + The name of the project that the new run will be created under. - An image input block used within EvalItem content arrays. + - `entity: optional string` - - `image_url: string` + The entity to use for the run. This allows you to set the team or username of the WandB user that you would + like associated with the run. If not set, the default entity for the registered WandB API key is used. - The URL of the image input. + - `name: optional string` - - `type: "input_image"` + A display name to set for the run. If not set, we will use the Job ID as the name. - The type of the image input. Always `input_image`. + - `tags: optional array of string` - - `"input_image"` + A list of tags to be attached to the newly created run. These tags are passed through directly to WandB. Some + default tags are generated by OpenAI: "openai/finetune", "openai/{base-model}", "openai/{ftjob-abcdef}". - - `detail: optional string` + - `metadata: optional Metadata` - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + Set of 16 key-value pairs that can be attached to an object. This can be + useful for storing additional information about the object in a structured + format, and querying for objects via API or the dashboard. - - `ResponseInputAudio = object { input_audio, type }` + Keys are strings with a maximum length of 64 characters. Values are strings + with a maximum length of 512 characters. - An audio input to the model. + - `method: optional object { type, dpo, reinforcement, supervised }` - - `input_audio: object { data, format }` + The method used for fine-tuning. - - `data: string` + - `type: "supervised" or "dpo" or "reinforcement"` - Base64-encoded audio data. + The type of method. Is either `supervised`, `dpo`, or `reinforcement`. - - `format: "mp3" or "wav"` + - `"supervised"` - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + - `"dpo"` - - `"mp3"` + - `"reinforcement"` - - `"wav"` + - `dpo: optional DpoMethod` - - `type: "input_audio"` + Configuration for the DPO fine-tuning method. - The type of the input item. Always `input_audio`. + - `hyperparameters: optional DpoHyperparameters` - - `"input_audio"` + The hyperparameters used for the DPO fine-tuning job. - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` + - `batch_size: optional "auto" or number` - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - `TextInput = string` + - `"auto"` - A text input to the model. + - `"auto"` - - `ResponseInputText = object { text, type }` + - `number` - A text input to the model. + - `beta: optional "auto" or number` - - `text: string` + The beta value for the DPO method. A higher beta value will increase the weight of the penalty between the policy and reference model. - The text input to the model. + - `"auto"` - - `type: "input_text"` + - `"auto"` - The type of the input item. Always `input_text`. + - `number` - - `"input_text"` + - `learning_rate_multiplier: optional "auto" or number` - - `OutputText = object { text, type }` + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - A text output from the model. + - `"auto"` - - `text: string` + - `"auto"` - The text output from the model. + - `number` - - `type: "output_text"` + - `n_epochs: optional "auto" or number` - The type of the output text. Always `output_text`. + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - `"output_text"` + - `"auto"` - - `InputImage = object { image_url, type, detail }` + - `"auto"` - An image input block used within EvalItem content arrays. + - `number` - - `image_url: string` + - `reinforcement: optional ReinforcementMethod` - The URL of the image input. + Configuration for the reinforcement fine-tuning method. - - `type: "input_image"` + - `grader: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - The type of the image input. Always `input_image`. + The grader used for the fine-tuning job. - - `"input_image"` + - `StringCheckGrader object { input, name, operation, 2 more }` - - `detail: optional string` + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + - `input: string` - - `ResponseInputAudio = object { input_audio, type }` + The input text. This may include template strings. - An audio input to the model. + - `name: string` - - `input_audio: object { data, format }` + The name of the grader. - - `data: string` + - `operation: "eq" or "ne" or "like" or "ilike"` - Base64-encoded audio data. + The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. - - `format: "mp3" or "wav"` + - `"eq"` - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + - `"ne"` - - `"mp3"` + - `"like"` - - `"wav"` + - `"ilike"` - - `type: "input_audio"` + - `reference: string` - The type of the input item. Always `input_audio`. + The reference text. This may include template strings. - - `"input_audio"` + - `type: "string_check"` - - `role: "user" or "assistant" or "system" or "developer"` + The object type, which is always `string_check`. - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. + - `"string_check"` - - `"user"` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` - - `"assistant"` + A TextSimilarityGrader object which grades text based on similarity metrics. - - `"system"` + - `evaluation_metric: "cosine" or "fuzzy_match" or "bleu" or 8 more` - - `"developer"` + The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, + `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, + or `rouge_l`. - - `type: optional "message"` + - `"cosine"` - The type of the message input. Always `message`. + - `"fuzzy_match"` - - `"message"` + - `"bleu"` - - `model: string` + - `"gleu"` - The model to use for the evaluation. + - `"meteor"` - - `name: string` + - `"rouge_1"` - The name of the grader. + - `"rouge_2"` - - `type: "score_model"` + - `"rouge_3"` - The object type, which is always `score_model`. + - `"rouge_4"` - - `"score_model"` + - `"rouge_5"` - - `range: optional array of number` + - `"rouge_l"` - The range of the score. Defaults to `[0, 1]`. + - `input: string` - - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` + The text being graded. - The sampling parameters for the model. + - `name: string` - - `max_completions_tokens: optional number` + The name of the grader. - The maximum number of tokens the grader model may generate in its response. + - `reference: string` - - `reasoning_effort: optional ReasoningEffort` + The text being graded against. - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. + - `type: "text_similarity"` - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + The type of grader. - - `"none"` + - `"text_similarity"` - - `"minimal"` + - `PythonGrader object { name, source, type, image_tag }` - - `"low"` + A PythonGrader object that runs a python script on the input. - - `"medium"` + - `name: string` - - `"high"` + The name of the grader. - - `"xhigh"` + - `source: string` - - `seed: optional number` + The source code of the python script. - A seed value to initialize the randomness, during sampling. + - `type: "python"` - - `temperature: optional number` + The object type, which is always `python`. - A higher temperature increases randomness in the outputs. + - `"python"` - - `top_p: optional number` + - `image_tag: optional string` - An alternative to temperature for nucleus sampling; 1.0 includes all tokens. + The image tag to use for the python script. - - `LabelModelGrader = object { input, labels, model, 3 more }` + - `ScoreModelGrader object { input, model, name, 3 more }` - A LabelModelGrader object which uses a model to assign labels to each item - in the evaluation. + A ScoreModelGrader object that uses a model to assign a score to the input. - `input: array of object { content, role, type }` + The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. + - `content: string or ResponseInputText or object { text, type } or 3 more` Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. @@ -12574,7 +7213,7 @@ curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/resume \ A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. @@ -12588,7 +7227,7 @@ curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/resume \ - `"input_text"` - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -12602,7 +7241,7 @@ curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/resume \ - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -12620,7 +7259,7 @@ curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/resume \ The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. @@ -12654,21 +7293,11 @@ curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/resume \ A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -12682,7 +7311,7 @@ curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/resume \ - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -12700,31 +7329,10 @@ curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/resume \ The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or @@ -12744,107 +7352,48 @@ curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/resume \ - `"message"` - - `labels: array of string` - - The labels to assign to each item in the evaluation. - - `model: string` - The model to use for the evaluation. Must support structured outputs. - - - `name: string` - - The name of the grader. - - - `passing_labels: array of string` - - The labels that indicate a passing result. Must be a subset of labels. - - - `type: "label_model"` - - The object type, which is always `label_model`. - - - `"label_model"` + The model to use for the evaluation. - `name: string` The name of the grader. - - `type: "multi"` - - The object type, which is always `multi`. - - - `"multi"` - - - `hyperparameters: optional ReinforcementHyperparameters` - - The hyperparameters used for the reinforcement fine-tuning job. - - - `batch_size: optional "auto" or number` - - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - - `"auto"` - - - `"auto"` - - - `number` - - - `compute_multiplier: optional "auto" or number` - - Multiplier on amount of compute used for exploring search space during training. - - - `"auto"` - - - `"auto"` - - - `number` - - - `eval_interval: optional "auto" or number` - - The number of training steps between evaluation runs. - - - `"auto"` - - - `"auto"` - - - `number` - - - `eval_samples: optional "auto" or number` - - Number of evaluation samples to generate per training step. - - - `"auto"` - - - `"auto"` - - - `number` + - `type: "score_model"` - - `learning_rate_multiplier: optional "auto" or number` + The object type, which is always `score_model`. - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. + - `"score_model"` - - `"auto"` + - `range: optional array of number` - - `"auto"` + The range of the score. Defaults to `[0, 1]`. - - `number` + - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` - - `n_epochs: optional "auto" or number` + The sampling parameters for the model. - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. + - `max_completions_tokens: optional number` - - `"auto"` + The maximum number of tokens the grader model may generate in its response. - - `"auto"` + - `reasoning_effort: optional ReasoningEffort` - - `number` + Constrains effort on reasoning for + [reasoning models](https://platform.openai.com/docs/guides/reasoning). + Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing + reasoning effort can result in faster responses and fewer tokens used + on reasoning in a response. - - `reasoning_effort: optional "default" or "low" or "medium" or "high"` + - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. + - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. + - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. + - `xhigh` is supported for all models after `gpt-5.1-codex-max`. - Level of reasoning effort. + - `"none"` - - `"default"` + - `"minimal"` - `"low"` @@ -12852,437 +7401,278 @@ curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/resume \ - `"high"` - - `supervised: optional SupervisedMethod` - - Configuration for the supervised fine-tuning method. - - - `hyperparameters: optional SupervisedHyperparameters` - - The hyperparameters used for the fine-tuning job. - - - `batch_size: optional "auto" or number` - - Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - - `"auto"` - - - `"auto"` - - - `number` - - - `learning_rate_multiplier: optional "auto" or number` - - Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - - `"auto"` - - - `"auto"` - - - `number` - - - `n_epochs: optional "auto" or number` - - The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - - `"auto"` - - - `"auto"` - - - `number` - -### Fine Tuning Job Event - -- `FineTuningJobEvent = object { id, created_at, level, 4 more }` - - Fine-tuning job event object - - - `id: string` - - The object identifier. - - - `created_at: number` - - The Unix timestamp (in seconds) for when the fine-tuning job was created. - - - `level: "info" or "warn" or "error"` - - The log level of the event. - - - `"info"` - - - `"warn"` - - - `"error"` - - - `message: string` - - The message of the event. - - - `object: "fine_tuning.job.event"` - - The object type, which is always "fine_tuning.job.event". - - - `"fine_tuning.job.event"` - - - `data: optional unknown` - - The data associated with the event. - - - `type: optional "message" or "metrics"` - - The type of event. - - - `"message"` - - - `"metrics"` - -### Fine Tuning Job Wandb Integration - -- `FineTuningJobWandbIntegration = object { project, entity, name, tags }` - - The settings for your integration with Weights and Biases. This payload specifies the project that - metrics will be sent to. Optionally, you can set an explicit display name for your run, add tags - to your run, and set a default entity (team, username, etc) to be associated with your run. - - - `project: string` + - `"xhigh"` - The name of the project that the new run will be created under. + - `seed: optional number` - - `entity: optional string` + A seed value to initialize the randomness, during sampling. - The entity to use for the run. This allows you to set the team or username of the WandB user that you would - like associated with the run. If not set, the default entity for the registered WandB API key is used. + - `temperature: optional number` - - `name: optional string` + A higher temperature increases randomness in the outputs. - A display name to set for the run. If not set, we will use the Job ID as the name. + - `top_p: optional number` - - `tags: optional array of string` + An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - A list of tags to be attached to the newly created run. These tags are passed through directly to WandB. Some - default tags are generated by OpenAI: "openai/finetune", "openai/{base-model}", "openai/{ftjob-abcdef}". + - `MultiGrader object { calculate_output, graders, name, type }` -### Fine Tuning Job Wandb Integration Object + A MultiGrader object combines the output of multiple graders to produce a single score. -- `FineTuningJobWandbIntegrationObject = object { type, wandb }` + - `calculate_output: string` - - `type: "wandb"` + A formula to calculate the output based on grader results. - The type of the integration being enabled for the fine-tuning job + - `graders: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - - `"wandb"` + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - `wandb: FineTuningJobWandbIntegration` + - `StringCheckGrader object { input, name, operation, 2 more }` - The settings for your integration with Weights and Biases. This payload specifies the project that - metrics will be sent to. Optionally, you can set an explicit display name for your run, add tags - to your run, and set a default entity (team, username, etc) to be associated with your run. + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - `project: string` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` - The name of the project that the new run will be created under. + A TextSimilarityGrader object which grades text based on similarity metrics. - - `entity: optional string` + - `PythonGrader object { name, source, type, image_tag }` - The entity to use for the run. This allows you to set the team or username of the WandB user that you would - like associated with the run. If not set, the default entity for the registered WandB API key is used. + A PythonGrader object that runs a python script on the input. - - `name: optional string` + - `ScoreModelGrader object { input, model, name, 3 more }` - A display name to set for the run. If not set, we will use the Job ID as the name. + A ScoreModelGrader object that uses a model to assign a score to the input. - - `tags: optional array of string` + - `LabelModelGrader object { input, labels, model, 3 more }` - A list of tags to be attached to the newly created run. These tags are passed through directly to WandB. Some - default tags are generated by OpenAI: "openai/finetune", "openai/{base-model}", "openai/{ftjob-abcdef}". + A LabelModelGrader object which uses a model to assign labels to each item + in the evaluation. -# Checkpoints + - `input: array of object { content, role, type }` -## List fine-tuning checkpoints + - `content: string or ResponseInputText or object { text, type } or 3 more` -**get** `/fine_tuning/jobs/{fine_tuning_job_id}/checkpoints` + Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. -List checkpoints for a fine-tuning job. + - `TextInput = string` -### Path Parameters + A text input to the model. -- `fine_tuning_job_id: string` + - `ResponseInputText object { text, type }` -### Query Parameters + A text input to the model. -- `after: optional string` + - `OutputText object { text, type }` - Identifier for the last checkpoint ID from the previous pagination request. + A text output from the model. -- `limit: optional number` + - `text: string` - Number of checkpoints to retrieve. + The text output from the model. -### Returns + - `type: "output_text"` -- `data: array of FineTuningJobCheckpoint` + The type of the output text. Always `output_text`. - - `id: string` + - `"output_text"` - The checkpoint identifier, which can be referenced in the API endpoints. + - `InputImage object { image_url, type, detail }` - - `created_at: number` + An image input block used within EvalItem content arrays. - The Unix timestamp (in seconds) for when the checkpoint was created. + - `image_url: string` - - `fine_tuned_model_checkpoint: string` + The URL of the image input. - The name of the fine-tuned checkpoint model that is created. + - `type: "input_image"` - - `fine_tuning_job_id: string` + The type of the image input. Always `input_image`. - The name of the fine-tuning job that this checkpoint was created from. + - `"input_image"` - - `metrics: object { full_valid_loss, full_valid_mean_token_accuracy, step, 4 more }` + - `detail: optional string` - Metrics at the step number during the fine-tuning job. + The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `full_valid_loss: optional number` + - `ResponseInputAudio object { input_audio, type }` - - `full_valid_mean_token_accuracy: optional number` + An audio input to the model. - - `step: optional number` + - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - `train_loss: optional number` + A list of inputs, each of which may be either an input text, output text, input + image, or input audio object. - - `train_mean_token_accuracy: optional number` + - `role: "user" or "assistant" or "system" or "developer"` - - `valid_loss: optional number` + The role of the message input. One of `user`, `assistant`, `system`, or + `developer`. - - `valid_mean_token_accuracy: optional number` + - `"user"` - - `object: "fine_tuning.job.checkpoint"` + - `"assistant"` - The object type, which is always "fine_tuning.job.checkpoint". + - `"system"` - - `"fine_tuning.job.checkpoint"` + - `"developer"` - - `step_number: number` + - `type: optional "message"` - The step number that the checkpoint was created at. + The type of the message input. Always `message`. -- `has_more: boolean` + - `"message"` -- `object: "list"` + - `labels: array of string` - - `"list"` + The labels to assign to each item in the evaluation. -- `first_id: optional string` + - `model: string` -- `last_id: optional string` + The model to use for the evaluation. Must support structured outputs. -### Example + - `name: string` -```http -curl https://api.openai.com/v1/fine_tuning/jobs/$FINE_TUNING_JOB_ID/checkpoints \ - -H "Authorization: Bearer $OPENAI_API_KEY" -``` + The name of the grader. -#### Response + - `passing_labels: array of string` -```json -{ - "data": [ - { - "id": "id", - "created_at": 0, - "fine_tuned_model_checkpoint": "fine_tuned_model_checkpoint", - "fine_tuning_job_id": "fine_tuning_job_id", - "metrics": { - "full_valid_loss": 0, - "full_valid_mean_token_accuracy": 0, - "step": 0, - "train_loss": 0, - "train_mean_token_accuracy": 0, - "valid_loss": 0, - "valid_mean_token_accuracy": 0 - }, - "object": "fine_tuning.job.checkpoint", - "step_number": 0 - } - ], - "has_more": true, - "object": "list", - "first_id": "first_id", - "last_id": "last_id" -} -``` + The labels that indicate a passing result. Must be a subset of labels. -### Example + - `type: "label_model"` -```http -curl https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/checkpoints \ - -H "Authorization: Bearer $OPENAI_API_KEY" -``` + The object type, which is always `label_model`. -#### Response + - `"label_model"` -```json -{ - "object": "list", - "data": [ - { - "object": "fine_tuning.job.checkpoint", - "id": "ftckpt_zc4Q7MP6XxulcVzj4MZdwsAB", - "created_at": 1721764867, - "fine_tuned_model_checkpoint": "ft:gpt-4o-mini-2024-07-18:my-org:custom-suffix:96olL566:ckpt-step-2000", - "metrics": { - "full_valid_loss": 0.134, - "full_valid_mean_token_accuracy": 0.874 - }, - "fine_tuning_job_id": "ftjob-abc123", - "step_number": 2000 - }, - { - "object": "fine_tuning.job.checkpoint", - "id": "ftckpt_enQCFmOTGj3syEpYVhBRLTSy", - "created_at": 1721764800, - "fine_tuned_model_checkpoint": "ft:gpt-4o-mini-2024-07-18:my-org:custom-suffix:7q8mpxmy:ckpt-step-1000", - "metrics": { - "full_valid_loss": 0.167, - "full_valid_mean_token_accuracy": 0.781 - }, - "fine_tuning_job_id": "ftjob-abc123", - "step_number": 1000 - } - ], - "first_id": "ftckpt_zc4Q7MP6XxulcVzj4MZdwsAB", - "last_id": "ftckpt_enQCFmOTGj3syEpYVhBRLTSy", - "has_more": true -} -``` + - `name: string` -## Domain Types + The name of the grader. -### Fine Tuning Job Checkpoint + - `type: "multi"` -- `FineTuningJobCheckpoint = object { id, created_at, fine_tuned_model_checkpoint, 4 more }` + The object type, which is always `multi`. - The `fine_tuning.job.checkpoint` object represents a model checkpoint for a fine-tuning job that is ready to use. + - `"multi"` - - `id: string` + - `hyperparameters: optional ReinforcementHyperparameters` - The checkpoint identifier, which can be referenced in the API endpoints. + The hyperparameters used for the reinforcement fine-tuning job. - - `created_at: number` + - `batch_size: optional "auto" or number` - The Unix timestamp (in seconds) for when the checkpoint was created. + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - - `fine_tuned_model_checkpoint: string` + - `"auto"` - The name of the fine-tuned checkpoint model that is created. + - `"auto"` - - `fine_tuning_job_id: string` + - `number` - The name of the fine-tuning job that this checkpoint was created from. + - `compute_multiplier: optional "auto" or number` - - `metrics: object { full_valid_loss, full_valid_mean_token_accuracy, step, 4 more }` + Multiplier on amount of compute used for exploring search space during training. - Metrics at the step number during the fine-tuning job. + - `"auto"` - - `full_valid_loss: optional number` + - `"auto"` - - `full_valid_mean_token_accuracy: optional number` + - `number` - - `step: optional number` + - `eval_interval: optional "auto" or number` - - `train_loss: optional number` + The number of training steps between evaluation runs. - - `train_mean_token_accuracy: optional number` + - `"auto"` - - `valid_loss: optional number` + - `"auto"` - - `valid_mean_token_accuracy: optional number` + - `number` - - `object: "fine_tuning.job.checkpoint"` + - `eval_samples: optional "auto" or number` - The object type, which is always "fine_tuning.job.checkpoint". + Number of evaluation samples to generate per training step. - - `"fine_tuning.job.checkpoint"` + - `"auto"` - - `step_number: number` + - `"auto"` - The step number that the checkpoint was created at. + - `number` -# Checkpoints + - `learning_rate_multiplier: optional "auto" or number` -# Permissions + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. -## List checkpoint permissions + - `"auto"` -**get** `/fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions` + - `"auto"` -**NOTE:** This endpoint requires an [admin API key](../admin-api-keys). + - `number` -Organization owners can use this endpoint to view all permissions for a fine-tuned model checkpoint. + - `n_epochs: optional "auto" or number` -### Path Parameters + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. -- `fine_tuned_model_checkpoint: string` + - `"auto"` -### Query Parameters + - `"auto"` -- `after: optional string` + - `number` - Identifier for the last permission ID from the previous pagination request. + - `reasoning_effort: optional "default" or "low" or "medium" or "high"` -- `limit: optional number` + Level of reasoning effort. - Number of permissions to retrieve. + - `"default"` -- `order: optional "ascending" or "descending"` + - `"low"` - The order in which to retrieve permissions. + - `"medium"` - - `"ascending"` + - `"high"` - - `"descending"` + - `supervised: optional SupervisedMethod` -- `project_id: optional string` + Configuration for the supervised fine-tuning method. - The ID of the project to get permissions for. + - `hyperparameters: optional SupervisedHyperparameters` -### Returns + The hyperparameters used for the fine-tuning job. -- `data: array of object { id, created_at, object, project_id }` + - `batch_size: optional "auto" or number` - - `id: string` + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - The permission identifier, which can be referenced in the API endpoints. + - `"auto"` - - `created_at: number` + - `"auto"` - The Unix timestamp (in seconds) for when the permission was created. + - `number` - - `object: "checkpoint.permission"` + - `learning_rate_multiplier: optional "auto" or number` - The object type, which is always "checkpoint.permission". + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. - - `"checkpoint.permission"` + - `"auto"` - - `project_id: string` + - `"auto"` - The project identifier that the permission is for. + - `number` -- `has_more: boolean` + - `n_epochs: optional "auto" or number` -- `object: "list"` + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - `"list"` + - `"auto"` -- `first_id: optional string` + - `"auto"` -- `last_id: optional string` + - `number` ### Example ```http -curl https://api.openai.com/v1/fine_tuning/checkpoints/$FINE_TUNED_MODEL_CHECKPOINT/permissions \ +curl https://api.openai.com/v1/fine_tuning/jobs/$FINE_TUNING_JOB_ID/resume \ + -X POST \ -H "Authorization: Bearer $OPENAI_API_KEY" ``` @@ -13290,25 +7680,91 @@ curl https://api.openai.com/v1/fine_tuning/checkpoints/$FINE_TUNED_MODEL_CHECKPO ```json { - "data": [ - { "id": "id", "created_at": 0, - "object": "checkpoint.permission", - "project_id": "project_id" + "error": { + "code": "code", + "message": "message", + "param": "param" + }, + "fine_tuned_model": "fine_tuned_model", + "finished_at": 0, + "hyperparameters": { + "batch_size": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto" + }, + "model": "model", + "object": "fine_tuning.job", + "organization_id": "organization_id", + "result_files": [ + "file-abc123" + ], + "seed": 0, + "status": "validating_files", + "trained_tokens": 0, + "training_file": "training_file", + "validation_file": "validation_file", + "estimated_finish": 0, + "integrations": [ + { + "type": "wandb", + "wandb": { + "project": "my-wandb-project", + "entity": "entity", + "name": "name", + "tags": [ + "custom-tag" + ] + } } ], - "has_more": true, - "object": "list", - "first_id": "first_id", - "last_id": "last_id" + "metadata": { + "foo": "string" + }, + "method": { + "type": "supervised", + "dpo": { + "hyperparameters": { + "batch_size": "auto", + "beta": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto" + } + }, + "reinforcement": { + "grader": { + "input": "input", + "name": "name", + "operation": "eq", + "reference": "reference", + "type": "string_check" + }, + "hyperparameters": { + "batch_size": "auto", + "compute_multiplier": "auto", + "eval_interval": "auto", + "eval_samples": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto", + "reasoning_effort": "default" + } + }, + "supervised": { + "hyperparameters": { + "batch_size": "auto", + "learning_rate_multiplier": "auto", + "n_epochs": "auto" + } + } + } } ``` ### Example ```http -curl https://api.openai.com/v1/fine_tuning/checkpoints/ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd/permissions \ +curl -X POST https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/resume \ -H "Authorization: Bearer $OPENAI_API_KEY" ``` @@ -13316,342 +7772,264 @@ curl https://api.openai.com/v1/fine_tuning/checkpoints/ft:gpt-4o-mini-2024-07-18 ```json { - "object": "list", - "data": [ - { - "object": "checkpoint.permission", - "id": "cp_zc4Q7MP6XxulcVzj4MZdwsAB", - "created_at": 1721764867, - "project_id": "proj_abGMw1llN8IrBb6SvvY5A1iH" - }, - { - "object": "checkpoint.permission", - "id": "cp_enQCFmOTGj3syEpYVhBRLTSy", + "object": "fine_tuning.job", + "id": "ftjob-abc123", + "model": "gpt-4o-mini-2024-07-18", "created_at": 1721764800, - "project_id": "proj_iqGMw1llN8IrBb6SvvY5A1oF" - }, - ], - "first_id": "cp_zc4Q7MP6XxulcVzj4MZdwsAB", - "last_id": "cp_enQCFmOTGj3syEpYVhBRLTSy", - "has_more": false + "fine_tuned_model": null, + "organization_id": "org-123", + "result_files": [], + "status": "queued", + "validation_file": "file-abc123", + "training_file": "file-abc123" } ``` -## List checkpoint permissions +## Domain Types -**get** `/fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions` +### Fine Tuning Job -**NOTE:** This endpoint requires an [admin API key](../admin-api-keys). +- `FineTuningJob object { id, created_at, error, 16 more }` -Organization owners can use this endpoint to view all permissions for a fine-tuned model checkpoint. + The `fine_tuning.job` object represents a fine-tuning job that has been created through the API. -### Path Parameters + - `id: string` -- `fine_tuned_model_checkpoint: string` + The object identifier, which can be referenced in the API endpoints. -### Query Parameters + - `created_at: number` -- `after: optional string` + The Unix timestamp (in seconds) for when the fine-tuning job was created. - Identifier for the last permission ID from the previous pagination request. + - `error: object { code, message, param }` -- `limit: optional number` + For fine-tuning jobs that have `failed`, this will contain more information on the cause of the failure. - Number of permissions to retrieve. + - `code: string` -- `order: optional "ascending" or "descending"` + A machine-readable error code. - The order in which to retrieve permissions. + - `message: string` - - `"ascending"` + A human-readable error message. - - `"descending"` + - `param: string` -- `project_id: optional string` + The parameter that was invalid, usually `training_file` or `validation_file`. This field will be null if the failure was not parameter-specific. - The ID of the project to get permissions for. + - `fine_tuned_model: string` -### Returns + The name of the fine-tuned model that is being created. The value will be null if the fine-tuning job is still running. -- `data: array of object { id, created_at, object, project_id }` + - `finished_at: number` - - `id: string` + The Unix timestamp (in seconds) for when the fine-tuning job was finished. The value will be null if the fine-tuning job is still running. - The permission identifier, which can be referenced in the API endpoints. + - `hyperparameters: object { batch_size, learning_rate_multiplier, n_epochs }` - - `created_at: number` + The hyperparameters used for the fine-tuning job. This value will only be returned when running `supervised` jobs. - The Unix timestamp (in seconds) for when the permission was created. + - `batch_size: optional "auto" or number` - - `object: "checkpoint.permission"` + Number of examples in each batch. A larger batch size means that model parameters + are updated less frequently, but with lower variance. - The object type, which is always "checkpoint.permission". + - `"auto"` - - `"checkpoint.permission"` + - `"auto"` + + - `number` + + - `learning_rate_multiplier: optional "auto" or number` + + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid + overfitting. + + - `"auto"` + + - `"auto"` + + - `number` + + - `n_epochs: optional "auto" or number` + + The number of epochs to train the model for. An epoch refers to one full cycle + through the training dataset. + + - `"auto"` + + - `"auto"` + + - `number` + + - `model: string` - - `project_id: string` + The base model that is being fine-tuned. - The project identifier that the permission is for. + - `object: "fine_tuning.job"` -- `has_more: boolean` + The object type, which is always "fine_tuning.job". -- `object: "list"` + - `"fine_tuning.job"` - - `"list"` + - `organization_id: string` -- `first_id: optional string` + The organization that owns the fine-tuning job. -- `last_id: optional string` + - `result_files: array of string` -### Example + The compiled results file ID(s) for the fine-tuning job. You can retrieve the results with the [Files API](/docs/api-reference/files/retrieve-contents). -```http -curl https://api.openai.com/v1/fine_tuning/checkpoints/$FINE_TUNED_MODEL_CHECKPOINT/permissions \ - -H "Authorization: Bearer $OPENAI_API_KEY" -``` + - `seed: number` -#### Response + The seed used for the fine-tuning job. -```json -{ - "data": [ - { - "id": "id", - "created_at": 0, - "object": "checkpoint.permission", - "project_id": "project_id" - } - ], - "has_more": true, - "object": "list", - "first_id": "first_id", - "last_id": "last_id" -} -``` + - `status: "validating_files" or "queued" or "running" or 3 more` -### Example + The current status of the fine-tuning job, which can be either `validating_files`, `queued`, `running`, `succeeded`, `failed`, or `cancelled`. -```http -curl https://api.openai.com/v1/fine_tuning/checkpoints/ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd/permissions \ - -H "Authorization: Bearer $OPENAI_API_KEY" -``` + - `"validating_files"` -#### Response + - `"queued"` -```json -{ - "object": "list", - "data": [ - { - "object": "checkpoint.permission", - "id": "cp_zc4Q7MP6XxulcVzj4MZdwsAB", - "created_at": 1721764867, - "project_id": "proj_abGMw1llN8IrBb6SvvY5A1iH" - }, - { - "object": "checkpoint.permission", - "id": "cp_enQCFmOTGj3syEpYVhBRLTSy", - "created_at": 1721764800, - "project_id": "proj_iqGMw1llN8IrBb6SvvY5A1oF" - }, - ], - "first_id": "cp_zc4Q7MP6XxulcVzj4MZdwsAB", - "last_id": "cp_enQCFmOTGj3syEpYVhBRLTSy", - "has_more": false -} -``` + - `"running"` -## Create checkpoint permissions + - `"succeeded"` -**post** `/fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions` + - `"failed"` -**NOTE:** Calling this endpoint requires an [admin API key](../admin-api-keys). + - `"cancelled"` -This enables organization owners to share fine-tuned models with other projects in their organization. + - `trained_tokens: number` -### Path Parameters + The total number of billable tokens processed by this fine-tuning job. The value will be null if the fine-tuning job is still running. -- `fine_tuned_model_checkpoint: string` + - `training_file: string` -### Body Parameters + The file ID used for training. You can retrieve the training data with the [Files API](/docs/api-reference/files/retrieve-contents). -- `project_ids: array of string` + - `validation_file: string` - The project identifiers to grant access to. + The file ID used for validation. You can retrieve the validation results with the [Files API](/docs/api-reference/files/retrieve-contents). -### Returns + - `estimated_finish: optional number` -- `data: array of object { id, created_at, object, project_id }` + The Unix timestamp (in seconds) for when the fine-tuning job is estimated to finish. The value will be null if the fine-tuning job is not running. - - `id: string` + - `integrations: optional array of FineTuningJobWandbIntegrationObject` - The permission identifier, which can be referenced in the API endpoints. + A list of integrations to enable for this fine-tuning job. - - `created_at: number` + - `type: "wandb"` - The Unix timestamp (in seconds) for when the permission was created. + The type of the integration being enabled for the fine-tuning job - - `object: "checkpoint.permission"` + - `"wandb"` - The object type, which is always "checkpoint.permission". + - `wandb: FineTuningJobWandbIntegration` - - `"checkpoint.permission"` + The settings for your integration with Weights and Biases. This payload specifies the project that + metrics will be sent to. Optionally, you can set an explicit display name for your run, add tags + to your run, and set a default entity (team, username, etc) to be associated with your run. - - `project_id: string` + - `project: string` - The project identifier that the permission is for. + The name of the project that the new run will be created under. -- `has_more: boolean` + - `entity: optional string` -- `object: "list"` + The entity to use for the run. This allows you to set the team or username of the WandB user that you would + like associated with the run. If not set, the default entity for the registered WandB API key is used. - - `"list"` + - `name: optional string` -- `first_id: optional string` + A display name to set for the run. If not set, we will use the Job ID as the name. -- `last_id: optional string` + - `tags: optional array of string` -### Example + A list of tags to be attached to the newly created run. These tags are passed through directly to WandB. Some + default tags are generated by OpenAI: "openai/finetune", "openai/{base-model}", "openai/{ftjob-abcdef}". -```http -curl https://api.openai.com/v1/fine_tuning/checkpoints/$FINE_TUNED_MODEL_CHECKPOINT/permissions \ - -H 'Content-Type: application/json' \ - -H "Authorization: Bearer $OPENAI_API_KEY" \ - -d '{ - "project_ids": [ - "string" - ] - }' -``` + - `metadata: optional Metadata` -#### Response + Set of 16 key-value pairs that can be attached to an object. This can be + useful for storing additional information about the object in a structured + format, and querying for objects via API or the dashboard. -```json -{ - "data": [ - { - "id": "id", - "created_at": 0, - "object": "checkpoint.permission", - "project_id": "project_id" - } - ], - "has_more": true, - "object": "list", - "first_id": "first_id", - "last_id": "last_id" -} -``` + Keys are strings with a maximum length of 64 characters. Values are strings + with a maximum length of 512 characters. -### Example + - `method: optional object { type, dpo, reinforcement, supervised }` -```http -curl https://api.openai.com/v1/fine_tuning/checkpoints/ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd/permissions \ - -H "Authorization: Bearer $OPENAI_API_KEY" - -d '{"project_ids": ["proj_abGMw1llN8IrBb6SvvY5A1iH"]}' -``` + The method used for fine-tuning. -#### Response + - `type: "supervised" or "dpo" or "reinforcement"` -```json -{ - "object": "list", - "data": [ - { - "object": "checkpoint.permission", - "id": "cp_zc4Q7MP6XxulcVzj4MZdwsAB", - "created_at": 1721764867, - "project_id": "proj_abGMw1llN8IrBb6SvvY5A1iH" - } - ], - "first_id": "cp_zc4Q7MP6XxulcVzj4MZdwsAB", - "last_id": "cp_zc4Q7MP6XxulcVzj4MZdwsAB", - "has_more": false -} -``` + The type of method. Is either `supervised`, `dpo`, or `reinforcement`. -## Delete checkpoint permission + - `"supervised"` -**delete** `/fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions/{permission_id}` + - `"dpo"` -**NOTE:** This endpoint requires an [admin API key](../admin-api-keys). + - `"reinforcement"` -Organization owners can use this endpoint to delete a permission for a fine-tuned model checkpoint. + - `dpo: optional DpoMethod` -### Path Parameters + Configuration for the DPO fine-tuning method. -- `fine_tuned_model_checkpoint: string` + - `hyperparameters: optional DpoHyperparameters` -- `permission_id: string` + The hyperparameters used for the DPO fine-tuning job. -### Returns + - `batch_size: optional "auto" or number` -- `id: string` + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. - The ID of the fine-tuned model checkpoint permission that was deleted. + - `"auto"` -- `deleted: boolean` + - `"auto"` - Whether the fine-tuned model checkpoint permission was successfully deleted. + - `number` -- `object: "checkpoint.permission"` + - `beta: optional "auto" or number` - The object type, which is always "checkpoint.permission". + The beta value for the DPO method. A higher beta value will increase the weight of the penalty between the policy and reference model. - - `"checkpoint.permission"` + - `"auto"` -### Example + - `"auto"` -```http -curl https://api.openai.com/v1/fine_tuning/checkpoints/$FINE_TUNED_MODEL_CHECKPOINT/permissions/$PERMISSION_ID \ - -X DELETE \ - -H "Authorization: Bearer $OPENAI_API_KEY" -``` + - `number` -#### Response + - `learning_rate_multiplier: optional "auto" or number` -```json -{ - "id": "id", - "deleted": true, - "object": "checkpoint.permission" -} -``` + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. -### Example + - `"auto"` -```http -curl https://api.openai.com/v1/fine_tuning/checkpoints/ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd/permissions/cp_zc4Q7MP6XxulcVzj4MZdwsAB \ - -H "Authorization: Bearer $OPENAI_API_KEY" -``` + - `"auto"` -#### Response + - `number` -```json -{ - "object": "checkpoint.permission", - "id": "cp_zc4Q7MP6XxulcVzj4MZdwsAB", - "deleted": true -} -``` + - `n_epochs: optional "auto" or number` -# Alpha + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. -# Graders + - `"auto"` -## Run grader + - `"auto"` -**post** `/fine_tuning/alpha/graders/run` + - `number` -Run a grader. + - `reinforcement: optional ReinforcementMethod` -### Body Parameters + Configuration for the reinforcement fine-tuning method. -- `grader: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` + - `grader: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` The grader used for the fine-tuning job. - - `StringCheckGrader = object { input, name, operation, 2 more }` + - `StringCheckGrader object { input, name, operation, 2 more }` A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. @@ -13685,7 +8063,7 @@ Run a grader. - `"string_check"` - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` A TextSimilarityGrader object which grades text based on similarity metrics. @@ -13735,7 +8113,7 @@ Run a grader. - `"text_similarity"` - - `PythonGrader = object { name, source, type, image_tag }` + - `PythonGrader object { name, source, type, image_tag }` A PythonGrader object that runs a python script on the input. @@ -13757,7 +8135,7 @@ Run a grader. The image tag to use for the python script. - - `ScoreModelGrader = object { input, model, name, 3 more }` + - `ScoreModelGrader object { input, model, name, 3 more }` A ScoreModelGrader object that uses a model to assign a score to the input. @@ -13773,7 +8151,7 @@ Run a grader. A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. @@ -13787,7 +8165,7 @@ Run a grader. - `"input_text"` - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -13801,7 +8179,7 @@ Run a grader. - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -13819,7 +8197,7 @@ Run a grader. The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. @@ -13853,21 +8231,11 @@ Run a grader. A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -13881,7 +8249,7 @@ Run a grader. - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -13899,31 +8267,10 @@ Run a grader. The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or @@ -14006,7 +8353,7 @@ Run a grader. An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `MultiGrader = object { calculate_output, graders, name, type }` + - `MultiGrader object { calculate_output, graders, name, type }` A MultiGrader object combines the output of multiple graders to produce a single score. @@ -14018,868 +8365,1103 @@ Run a grader. A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - `StringCheckGrader = object { input, name, operation, 2 more }` + - `StringCheckGrader object { input, name, operation, 2 more }` A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - `input: string` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` - The input text. This may include template strings. + A TextSimilarityGrader object which grades text based on similarity metrics. + + - `PythonGrader object { name, source, type, image_tag }` + + A PythonGrader object that runs a python script on the input. + + - `ScoreModelGrader object { input, model, name, 3 more }` + + A ScoreModelGrader object that uses a model to assign a score to the input. + + - `LabelModelGrader object { input, labels, model, 3 more }` + + A LabelModelGrader object which uses a model to assign labels to each item + in the evaluation. + + - `input: array of object { content, role, type }` + + - `content: string or ResponseInputText or object { text, type } or 3 more` + + Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. + + - `TextInput = string` + + A text input to the model. + + - `ResponseInputText object { text, type }` + + A text input to the model. + + - `OutputText object { text, type }` + + A text output from the model. + + - `text: string` + + The text output from the model. + + - `type: "output_text"` + + The type of the output text. Always `output_text`. + + - `"output_text"` + + - `InputImage object { image_url, type, detail }` + + An image input block used within EvalItem content arrays. + + - `image_url: string` + + The URL of the image input. + + - `type: "input_image"` + + The type of the image input. Always `input_image`. + + - `"input_image"` + + - `detail: optional string` + + The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + + - `ResponseInputAudio object { input_audio, type }` + + An audio input to the model. + + - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` + + A list of inputs, each of which may be either an input text, output text, input + image, or input audio object. + + - `role: "user" or "assistant" or "system" or "developer"` + + The role of the message input. One of `user`, `assistant`, `system`, or + `developer`. + + - `"user"` + + - `"assistant"` + + - `"system"` + + - `"developer"` + + - `type: optional "message"` + + The type of the message input. Always `message`. + + - `"message"` + + - `labels: array of string` + + The labels to assign to each item in the evaluation. + + - `model: string` + + The model to use for the evaluation. Must support structured outputs. - `name: string` The name of the grader. - - `operation: "eq" or "ne" or "like" or "ilike"` + - `passing_labels: array of string` + + The labels that indicate a passing result. Must be a subset of labels. + + - `type: "label_model"` + + The object type, which is always `label_model`. + + - `"label_model"` + + - `name: string` + + The name of the grader. + + - `type: "multi"` + + The object type, which is always `multi`. + + - `"multi"` + + - `hyperparameters: optional ReinforcementHyperparameters` + + The hyperparameters used for the reinforcement fine-tuning job. + + - `batch_size: optional "auto" or number` + + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. + + - `"auto"` + + - `"auto"` + + - `number` + + - `compute_multiplier: optional "auto" or number` + + Multiplier on amount of compute used for exploring search space during training. + + - `"auto"` + + - `"auto"` + + - `number` + + - `eval_interval: optional "auto" or number` + + The number of training steps between evaluation runs. + + - `"auto"` + + - `"auto"` + + - `number` + + - `eval_samples: optional "auto" or number` + + Number of evaluation samples to generate per training step. + + - `"auto"` + + - `"auto"` + + - `number` + + - `learning_rate_multiplier: optional "auto" or number` + + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. + + - `"auto"` + + - `"auto"` + + - `number` + + - `n_epochs: optional "auto" or number` + + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. + + - `"auto"` + + - `"auto"` + + - `number` + + - `reasoning_effort: optional "default" or "low" or "medium" or "high"` + + Level of reasoning effort. + + - `"default"` + + - `"low"` + + - `"medium"` + + - `"high"` + + - `supervised: optional SupervisedMethod` + + Configuration for the supervised fine-tuning method. + + - `hyperparameters: optional SupervisedHyperparameters` + + The hyperparameters used for the fine-tuning job. + + - `batch_size: optional "auto" or number` + + Number of examples in each batch. A larger batch size means that model parameters are updated less frequently, but with lower variance. + + - `"auto"` + + - `"auto"` + + - `number` + + - `learning_rate_multiplier: optional "auto" or number` + + Scaling factor for the learning rate. A smaller learning rate may be useful to avoid overfitting. + + - `"auto"` + + - `"auto"` - The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. + - `number` - - `"eq"` + - `n_epochs: optional "auto" or number` - - `"ne"` + The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset. - - `"like"` + - `"auto"` - - `"ilike"` + - `"auto"` - - `reference: string` + - `number` - The reference text. This may include template strings. +### Fine Tuning Job Event - - `type: "string_check"` +- `FineTuningJobEvent object { id, created_at, level, 4 more }` - The object type, which is always `string_check`. + Fine-tuning job event object - - `"string_check"` + - `id: string` - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` + The object identifier. - A TextSimilarityGrader object which grades text based on similarity metrics. + - `created_at: number` - - `evaluation_metric: "cosine" or "fuzzy_match" or "bleu" or 8 more` + The Unix timestamp (in seconds) for when the fine-tuning job was created. - The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, - `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, - or `rouge_l`. + - `level: "info" or "warn" or "error"` - - `"cosine"` + The log level of the event. - - `"fuzzy_match"` + - `"info"` - - `"bleu"` + - `"warn"` - - `"gleu"` + - `"error"` - - `"meteor"` + - `message: string` - - `"rouge_1"` + The message of the event. - - `"rouge_2"` + - `object: "fine_tuning.job.event"` - - `"rouge_3"` + The object type, which is always "fine_tuning.job.event". - - `"rouge_4"` + - `"fine_tuning.job.event"` - - `"rouge_5"` + - `data: optional unknown` - - `"rouge_l"` + The data associated with the event. - - `input: string` + - `type: optional "message" or "metrics"` - The text being graded. + The type of event. - - `name: string` + - `"message"` - The name of the grader. + - `"metrics"` - - `reference: string` +### Fine Tuning Job Wandb Integration - The text being graded against. +- `FineTuningJobWandbIntegration object { project, entity, name, tags }` - - `type: "text_similarity"` + The settings for your integration with Weights and Biases. This payload specifies the project that + metrics will be sent to. Optionally, you can set an explicit display name for your run, add tags + to your run, and set a default entity (team, username, etc) to be associated with your run. - The type of grader. + - `project: string` - - `"text_similarity"` + The name of the project that the new run will be created under. - - `PythonGrader = object { name, source, type, image_tag }` + - `entity: optional string` - A PythonGrader object that runs a python script on the input. + The entity to use for the run. This allows you to set the team or username of the WandB user that you would + like associated with the run. If not set, the default entity for the registered WandB API key is used. - - `name: string` + - `name: optional string` - The name of the grader. + A display name to set for the run. If not set, we will use the Job ID as the name. - - `source: string` + - `tags: optional array of string` - The source code of the python script. + A list of tags to be attached to the newly created run. These tags are passed through directly to WandB. Some + default tags are generated by OpenAI: "openai/finetune", "openai/{base-model}", "openai/{ftjob-abcdef}". - - `type: "python"` +### Fine Tuning Job Wandb Integration Object - The object type, which is always `python`. +- `FineTuningJobWandbIntegrationObject object { type, wandb }` - - `"python"` + - `type: "wandb"` - - `image_tag: optional string` + The type of the integration being enabled for the fine-tuning job - The image tag to use for the python script. + - `"wandb"` - - `ScoreModelGrader = object { input, model, name, 3 more }` + - `wandb: FineTuningJobWandbIntegration` - A ScoreModelGrader object that uses a model to assign a score to the input. + The settings for your integration with Weights and Biases. This payload specifies the project that + metrics will be sent to. Optionally, you can set an explicit display name for your run, add tags + to your run, and set a default entity (team, username, etc) to be associated with your run. - - `input: array of object { content, role, type }` + - `project: string` - The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. + The name of the project that the new run will be created under. - - `content: string or ResponseInputText or object { text, type } or 3 more` + - `entity: optional string` - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. + The entity to use for the run. This allows you to set the team or username of the WandB user that you would + like associated with the run. If not set, the default entity for the registered WandB API key is used. - - `TextInput = string` + - `name: optional string` - A text input to the model. + A display name to set for the run. If not set, we will use the Job ID as the name. - - `ResponseInputText = object { text, type }` + - `tags: optional array of string` - A text input to the model. + A list of tags to be attached to the newly created run. These tags are passed through directly to WandB. Some + default tags are generated by OpenAI: "openai/finetune", "openai/{base-model}", "openai/{ftjob-abcdef}". - - `text: string` +# Checkpoints - The text input to the model. +## List fine-tuning checkpoints - - `type: "input_text"` +**get** `/fine_tuning/jobs/{fine_tuning_job_id}/checkpoints` - The type of the input item. Always `input_text`. +List checkpoints for a fine-tuning job. - - `"input_text"` +### Path Parameters - - `OutputText = object { text, type }` +- `fine_tuning_job_id: string` - A text output from the model. +### Query Parameters - - `text: string` +- `after: optional string` - The text output from the model. + Identifier for the last checkpoint ID from the previous pagination request. - - `type: "output_text"` +- `limit: optional number` - The type of the output text. Always `output_text`. + Number of checkpoints to retrieve. - - `"output_text"` +### Returns - - `InputImage = object { image_url, type, detail }` +- `data: array of FineTuningJobCheckpoint` - An image input block used within EvalItem content arrays. + - `id: string` - - `image_url: string` + The checkpoint identifier, which can be referenced in the API endpoints. - The URL of the image input. + - `created_at: number` - - `type: "input_image"` + The Unix timestamp (in seconds) for when the checkpoint was created. - The type of the image input. Always `input_image`. + - `fine_tuned_model_checkpoint: string` - - `"input_image"` + The name of the fine-tuned checkpoint model that is created. - - `detail: optional string` + - `fine_tuning_job_id: string` - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + The name of the fine-tuning job that this checkpoint was created from. - - `ResponseInputAudio = object { input_audio, type }` + - `metrics: object { full_valid_loss, full_valid_mean_token_accuracy, step, 4 more }` - An audio input to the model. + Metrics at the step number during the fine-tuning job. - - `input_audio: object { data, format }` + - `full_valid_loss: optional number` - - `data: string` + - `full_valid_mean_token_accuracy: optional number` - Base64-encoded audio data. + - `step: optional number` - - `format: "mp3" or "wav"` + - `train_loss: optional number` - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + - `train_mean_token_accuracy: optional number` - - `"mp3"` + - `valid_loss: optional number` - - `"wav"` + - `valid_mean_token_accuracy: optional number` - - `type: "input_audio"` + - `object: "fine_tuning.job.checkpoint"` - The type of the input item. Always `input_audio`. + The object type, which is always "fine_tuning.job.checkpoint". - - `"input_audio"` + - `"fine_tuning.job.checkpoint"` - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` + - `step_number: number` - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. + The step number that the checkpoint was created at. - - `TextInput = string` +- `has_more: boolean` - A text input to the model. +- `object: "list"` - - `ResponseInputText = object { text, type }` + - `"list"` - A text input to the model. +- `first_id: optional string` - - `text: string` +- `last_id: optional string` - The text input to the model. +### Example - - `type: "input_text"` +```http +curl https://api.openai.com/v1/fine_tuning/jobs/$FINE_TUNING_JOB_ID/checkpoints \ + -H "Authorization: Bearer $OPENAI_API_KEY" +``` - The type of the input item. Always `input_text`. +#### Response - - `"input_text"` +```json +{ + "data": [ + { + "id": "id", + "created_at": 0, + "fine_tuned_model_checkpoint": "fine_tuned_model_checkpoint", + "fine_tuning_job_id": "fine_tuning_job_id", + "metrics": { + "full_valid_loss": 0, + "full_valid_mean_token_accuracy": 0, + "step": 0, + "train_loss": 0, + "train_mean_token_accuracy": 0, + "valid_loss": 0, + "valid_mean_token_accuracy": 0 + }, + "object": "fine_tuning.job.checkpoint", + "step_number": 0 + } + ], + "has_more": true, + "object": "list", + "first_id": "first_id", + "last_id": "last_id" +} +``` - - `OutputText = object { text, type }` +### Example - A text output from the model. +```http +curl https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/checkpoints \ + -H "Authorization: Bearer $OPENAI_API_KEY" +``` - - `text: string` +#### Response - The text output from the model. +```json +{ + "object": "list", + "data": [ + { + "object": "fine_tuning.job.checkpoint", + "id": "ftckpt_zc4Q7MP6XxulcVzj4MZdwsAB", + "created_at": 1721764867, + "fine_tuned_model_checkpoint": "ft:gpt-4o-mini-2024-07-18:my-org:custom-suffix:96olL566:ckpt-step-2000", + "metrics": { + "full_valid_loss": 0.134, + "full_valid_mean_token_accuracy": 0.874 + }, + "fine_tuning_job_id": "ftjob-abc123", + "step_number": 2000 + }, + { + "object": "fine_tuning.job.checkpoint", + "id": "ftckpt_enQCFmOTGj3syEpYVhBRLTSy", + "created_at": 1721764800, + "fine_tuned_model_checkpoint": "ft:gpt-4o-mini-2024-07-18:my-org:custom-suffix:7q8mpxmy:ckpt-step-1000", + "metrics": { + "full_valid_loss": 0.167, + "full_valid_mean_token_accuracy": 0.781 + }, + "fine_tuning_job_id": "ftjob-abc123", + "step_number": 1000 + } + ], + "first_id": "ftckpt_zc4Q7MP6XxulcVzj4MZdwsAB", + "last_id": "ftckpt_enQCFmOTGj3syEpYVhBRLTSy", + "has_more": true +} +``` - - `type: "output_text"` +## Domain Types - The type of the output text. Always `output_text`. +### Fine Tuning Job Checkpoint - - `"output_text"` +- `FineTuningJobCheckpoint object { id, created_at, fine_tuned_model_checkpoint, 4 more }` - - `InputImage = object { image_url, type, detail }` + The `fine_tuning.job.checkpoint` object represents a model checkpoint for a fine-tuning job that is ready to use. - An image input block used within EvalItem content arrays. + - `id: string` - - `image_url: string` + The checkpoint identifier, which can be referenced in the API endpoints. - The URL of the image input. + - `created_at: number` - - `type: "input_image"` + The Unix timestamp (in seconds) for when the checkpoint was created. - The type of the image input. Always `input_image`. + - `fine_tuned_model_checkpoint: string` - - `"input_image"` + The name of the fine-tuned checkpoint model that is created. - - `detail: optional string` + - `fine_tuning_job_id: string` - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + The name of the fine-tuning job that this checkpoint was created from. - - `ResponseInputAudio = object { input_audio, type }` + - `metrics: object { full_valid_loss, full_valid_mean_token_accuracy, step, 4 more }` - An audio input to the model. + Metrics at the step number during the fine-tuning job. - - `input_audio: object { data, format }` + - `full_valid_loss: optional number` - - `data: string` + - `full_valid_mean_token_accuracy: optional number` - Base64-encoded audio data. + - `step: optional number` - - `format: "mp3" or "wav"` + - `train_loss: optional number` - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + - `train_mean_token_accuracy: optional number` - - `"mp3"` + - `valid_loss: optional number` - - `"wav"` + - `valid_mean_token_accuracy: optional number` - - `type: "input_audio"` + - `object: "fine_tuning.job.checkpoint"` - The type of the input item. Always `input_audio`. + The object type, which is always "fine_tuning.job.checkpoint". - - `"input_audio"` + - `"fine_tuning.job.checkpoint"` - - `role: "user" or "assistant" or "system" or "developer"` + - `step_number: number` - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. + The step number that the checkpoint was created at. - - `"user"` +# Checkpoints - - `"assistant"` +# Permissions - - `"system"` +## List checkpoint permissions - - `"developer"` +**get** `/fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions` - - `type: optional "message"` +**NOTE:** This endpoint requires an [admin API key](../admin-api-keys). - The type of the message input. Always `message`. +Organization owners can use this endpoint to view all permissions for a fine-tuned model checkpoint. - - `"message"` +### Path Parameters - - `model: string` +- `fine_tuned_model_checkpoint: string` - The model to use for the evaluation. +### Query Parameters - - `name: string` +- `after: optional string` - The name of the grader. + Identifier for the last permission ID from the previous pagination request. - - `type: "score_model"` +- `limit: optional number` - The object type, which is always `score_model`. + Number of permissions to retrieve. - - `"score_model"` +- `order: optional "ascending" or "descending"` - - `range: optional array of number` + The order in which to retrieve permissions. - The range of the score. Defaults to `[0, 1]`. + - `"ascending"` - - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` + - `"descending"` - The sampling parameters for the model. +- `project_id: optional string` - - `max_completions_tokens: optional number` + The ID of the project to get permissions for. - The maximum number of tokens the grader model may generate in its response. +### Returns - - `reasoning_effort: optional ReasoningEffort` +- `data: array of object { id, created_at, object, project_id }` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. + - `id: string` - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + The permission identifier, which can be referenced in the API endpoints. - - `"none"` + - `created_at: number` - - `"minimal"` + The Unix timestamp (in seconds) for when the permission was created. - - `"low"` + - `object: "checkpoint.permission"` - - `"medium"` + The object type, which is always "checkpoint.permission". - - `"high"` + - `"checkpoint.permission"` - - `"xhigh"` + - `project_id: string` - - `seed: optional number` + The project identifier that the permission is for. - A seed value to initialize the randomness, during sampling. +- `has_more: boolean` - - `temperature: optional number` +- `object: "list"` - A higher temperature increases randomness in the outputs. + - `"list"` - - `top_p: optional number` +- `first_id: optional string` - An alternative to temperature for nucleus sampling; 1.0 includes all tokens. +- `last_id: optional string` - - `LabelModelGrader = object { input, labels, model, 3 more }` +### Example - A LabelModelGrader object which uses a model to assign labels to each item - in the evaluation. +```http +curl https://api.openai.com/v1/fine_tuning/checkpoints/$FINE_TUNED_MODEL_CHECKPOINT/permissions \ + -H "Authorization: Bearer $OPENAI_API_KEY" +``` - - `input: array of object { content, role, type }` +#### Response - - `content: string or ResponseInputText or object { text, type } or 3 more` +```json +{ + "data": [ + { + "id": "id", + "created_at": 0, + "object": "checkpoint.permission", + "project_id": "project_id" + } + ], + "has_more": true, + "object": "list", + "first_id": "first_id", + "last_id": "last_id" +} +``` - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. +### Example - - `TextInput = string` +```http +curl https://api.openai.com/v1/fine_tuning/checkpoints/ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd/permissions \ + -H "Authorization: Bearer $OPENAI_API_KEY" +``` - A text input to the model. +#### Response - - `ResponseInputText = object { text, type }` +```json +{ + "object": "list", + "data": [ + { + "object": "checkpoint.permission", + "id": "cp_zc4Q7MP6XxulcVzj4MZdwsAB", + "created_at": 1721764867, + "project_id": "proj_abGMw1llN8IrBb6SvvY5A1iH" + }, + { + "object": "checkpoint.permission", + "id": "cp_enQCFmOTGj3syEpYVhBRLTSy", + "created_at": 1721764800, + "project_id": "proj_iqGMw1llN8IrBb6SvvY5A1oF" + }, + ], + "first_id": "cp_zc4Q7MP6XxulcVzj4MZdwsAB", + "last_id": "cp_enQCFmOTGj3syEpYVhBRLTSy", + "has_more": false +} +``` - A text input to the model. +## List checkpoint permissions - - `text: string` +**get** `/fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions` - The text input to the model. +**NOTE:** This endpoint requires an [admin API key](../admin-api-keys). - - `type: "input_text"` +Organization owners can use this endpoint to view all permissions for a fine-tuned model checkpoint. - The type of the input item. Always `input_text`. +### Path Parameters - - `"input_text"` +- `fine_tuned_model_checkpoint: string` - - `OutputText = object { text, type }` +### Query Parameters - A text output from the model. +- `after: optional string` - - `text: string` + Identifier for the last permission ID from the previous pagination request. - The text output from the model. +- `limit: optional number` - - `type: "output_text"` + Number of permissions to retrieve. - The type of the output text. Always `output_text`. +- `order: optional "ascending" or "descending"` - - `"output_text"` + The order in which to retrieve permissions. - - `InputImage = object { image_url, type, detail }` + - `"ascending"` - An image input block used within EvalItem content arrays. + - `"descending"` - - `image_url: string` +- `project_id: optional string` - The URL of the image input. + The ID of the project to get permissions for. - - `type: "input_image"` +### Returns - The type of the image input. Always `input_image`. +- `data: array of object { id, created_at, object, project_id }` - - `"input_image"` + - `id: string` - - `detail: optional string` + The permission identifier, which can be referenced in the API endpoints. - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + - `created_at: number` - - `ResponseInputAudio = object { input_audio, type }` + The Unix timestamp (in seconds) for when the permission was created. - An audio input to the model. + - `object: "checkpoint.permission"` - - `input_audio: object { data, format }` + The object type, which is always "checkpoint.permission". - - `data: string` + - `"checkpoint.permission"` - Base64-encoded audio data. + - `project_id: string` - - `format: "mp3" or "wav"` + The project identifier that the permission is for. - The format of the audio data. Currently supported formats are `mp3` and - `wav`. +- `has_more: boolean` - - `"mp3"` +- `object: "list"` - - `"wav"` + - `"list"` - - `type: "input_audio"` +- `first_id: optional string` - The type of the input item. Always `input_audio`. +- `last_id: optional string` - - `"input_audio"` +### Example - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` +```http +curl https://api.openai.com/v1/fine_tuning/checkpoints/$FINE_TUNED_MODEL_CHECKPOINT/permissions \ + -H "Authorization: Bearer $OPENAI_API_KEY" +``` - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. +#### Response - - `TextInput = string` +```json +{ + "data": [ + { + "id": "id", + "created_at": 0, + "object": "checkpoint.permission", + "project_id": "project_id" + } + ], + "has_more": true, + "object": "list", + "first_id": "first_id", + "last_id": "last_id" +} +``` - A text input to the model. +### Example - - `ResponseInputText = object { text, type }` +```http +curl https://api.openai.com/v1/fine_tuning/checkpoints/ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd/permissions \ + -H "Authorization: Bearer $OPENAI_API_KEY" +``` - A text input to the model. +#### Response - - `text: string` +```json +{ + "object": "list", + "data": [ + { + "object": "checkpoint.permission", + "id": "cp_zc4Q7MP6XxulcVzj4MZdwsAB", + "created_at": 1721764867, + "project_id": "proj_abGMw1llN8IrBb6SvvY5A1iH" + }, + { + "object": "checkpoint.permission", + "id": "cp_enQCFmOTGj3syEpYVhBRLTSy", + "created_at": 1721764800, + "project_id": "proj_iqGMw1llN8IrBb6SvvY5A1oF" + }, + ], + "first_id": "cp_zc4Q7MP6XxulcVzj4MZdwsAB", + "last_id": "cp_enQCFmOTGj3syEpYVhBRLTSy", + "has_more": false +} +``` - The text input to the model. +## Create checkpoint permissions - - `type: "input_text"` +**post** `/fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions` - The type of the input item. Always `input_text`. +**NOTE:** Calling this endpoint requires an [admin API key](../admin-api-keys). - - `"input_text"` +This enables organization owners to share fine-tuned models with other projects in their organization. - - `OutputText = object { text, type }` +### Path Parameters - A text output from the model. +- `fine_tuned_model_checkpoint: string` - - `text: string` +### Body Parameters - The text output from the model. +- `project_ids: array of string` - - `type: "output_text"` + The project identifiers to grant access to. - The type of the output text. Always `output_text`. +### Returns - - `"output_text"` +- `data: array of object { id, created_at, object, project_id }` - - `InputImage = object { image_url, type, detail }` + - `id: string` - An image input block used within EvalItem content arrays. + The permission identifier, which can be referenced in the API endpoints. - - `image_url: string` + - `created_at: number` - The URL of the image input. + The Unix timestamp (in seconds) for when the permission was created. - - `type: "input_image"` + - `object: "checkpoint.permission"` - The type of the image input. Always `input_image`. + The object type, which is always "checkpoint.permission". - - `"input_image"` + - `"checkpoint.permission"` - - `detail: optional string` + - `project_id: string` - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + The project identifier that the permission is for. - - `ResponseInputAudio = object { input_audio, type }` +- `has_more: boolean` - An audio input to the model. +- `object: "list"` - - `input_audio: object { data, format }` + - `"list"` - - `data: string` +- `first_id: optional string` - Base64-encoded audio data. +- `last_id: optional string` - - `format: "mp3" or "wav"` +### Example - The format of the audio data. Currently supported formats are `mp3` and - `wav`. +```http +curl https://api.openai.com/v1/fine_tuning/checkpoints/$FINE_TUNED_MODEL_CHECKPOINT/permissions \ + -H 'Content-Type: application/json' \ + -H "Authorization: Bearer $OPENAI_API_KEY" \ + -d '{ + "project_ids": [ + "string" + ] + }' +``` - - `"mp3"` +#### Response - - `"wav"` +```json +{ + "data": [ + { + "id": "id", + "created_at": 0, + "object": "checkpoint.permission", + "project_id": "project_id" + } + ], + "has_more": true, + "object": "list", + "first_id": "first_id", + "last_id": "last_id" +} +``` - - `type: "input_audio"` +### Example - The type of the input item. Always `input_audio`. +```http +curl https://api.openai.com/v1/fine_tuning/checkpoints/ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd/permissions \ + -H "Authorization: Bearer $OPENAI_API_KEY" + -d '{"project_ids": ["proj_abGMw1llN8IrBb6SvvY5A1iH"]}' +``` - - `"input_audio"` +#### Response - - `role: "user" or "assistant" or "system" or "developer"` +```json +{ + "object": "list", + "data": [ + { + "object": "checkpoint.permission", + "id": "cp_zc4Q7MP6XxulcVzj4MZdwsAB", + "created_at": 1721764867, + "project_id": "proj_abGMw1llN8IrBb6SvvY5A1iH" + } + ], + "first_id": "cp_zc4Q7MP6XxulcVzj4MZdwsAB", + "last_id": "cp_zc4Q7MP6XxulcVzj4MZdwsAB", + "has_more": false +} +``` - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. +## Delete checkpoint permission - - `"user"` +**delete** `/fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions/{permission_id}` - - `"assistant"` +**NOTE:** This endpoint requires an [admin API key](../admin-api-keys). - - `"system"` +Organization owners can use this endpoint to delete a permission for a fine-tuned model checkpoint. - - `"developer"` +### Path Parameters - - `type: optional "message"` +- `fine_tuned_model_checkpoint: string` - The type of the message input. Always `message`. +- `permission_id: string` - - `"message"` +### Returns - - `labels: array of string` +- `id: string` - The labels to assign to each item in the evaluation. + The ID of the fine-tuned model checkpoint permission that was deleted. - - `model: string` +- `deleted: boolean` - The model to use for the evaluation. Must support structured outputs. + Whether the fine-tuned model checkpoint permission was successfully deleted. - - `name: string` +- `object: "checkpoint.permission"` - The name of the grader. + The object type, which is always "checkpoint.permission". - - `passing_labels: array of string` + - `"checkpoint.permission"` - The labels that indicate a passing result. Must be a subset of labels. +### Example - - `type: "label_model"` +```http +curl https://api.openai.com/v1/fine_tuning/checkpoints/$FINE_TUNED_MODEL_CHECKPOINT/permissions/$PERMISSION_ID \ + -X DELETE \ + -H "Authorization: Bearer $OPENAI_API_KEY" +``` - The object type, which is always `label_model`. +#### Response - - `"label_model"` +```json +{ + "id": "id", + "deleted": true, + "object": "checkpoint.permission" +} +``` - - `name: string` +### Example - The name of the grader. +```http +curl https://api.openai.com/v1/fine_tuning/checkpoints/ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd/permissions/cp_zc4Q7MP6XxulcVzj4MZdwsAB \ + -H "Authorization: Bearer $OPENAI_API_KEY" +``` - - `type: "multi"` +#### Response - The object type, which is always `multi`. +```json +{ + "object": "checkpoint.permission", + "id": "cp_zc4Q7MP6XxulcVzj4MZdwsAB", + "deleted": true +} +``` - - `"multi"` +## Domain Types -- `model_sample: string` +### Permission Retrieve Response - The model sample to be evaluated. This value will be used to populate - the `sample` namespace. See [the guide](/docs/guides/graders) for more details. - The `output_json` variable will be populated if the model sample is a - valid JSON string. +- `PermissionRetrieveResponse object { data, has_more, object, 2 more }` -- `item: optional unknown` + - `data: array of object { id, created_at, object, project_id }` - The dataset item provided to the grader. This will be used to populate - the `item` namespace. See [the guide](/docs/guides/graders) for more details. + - `id: string` -### Returns + The permission identifier, which can be referenced in the API endpoints. -- `metadata: object { errors, execution_time, name, 4 more }` + - `created_at: number` - - `errors: object { formula_parse_error, invalid_variable_error, model_grader_parse_error, 11 more }` + The Unix timestamp (in seconds) for when the permission was created. - - `formula_parse_error: boolean` + - `object: "checkpoint.permission"` - - `invalid_variable_error: boolean` + The object type, which is always "checkpoint.permission". - - `model_grader_parse_error: boolean` + - `"checkpoint.permission"` - - `model_grader_refusal_error: boolean` + - `project_id: string` - - `model_grader_server_error: boolean` + The project identifier that the permission is for. - - `model_grader_server_error_details: string` + - `has_more: boolean` - - `other_error: boolean` + - `object: "list"` - - `python_grader_runtime_error: boolean` + - `"list"` - - `python_grader_runtime_error_details: string` + - `first_id: optional string` - - `python_grader_server_error: boolean` + - `last_id: optional string` - - `python_grader_server_error_type: string` +### Permission List Response - - `sample_parse_error: boolean` +- `PermissionListResponse object { id, created_at, object, project_id }` - - `truncated_observation_error: boolean` + The `checkpoint.permission` object represents a permission for a fine-tuned model checkpoint. - - `unresponsive_reward_error: boolean` + - `id: string` - - `execution_time: number` + The permission identifier, which can be referenced in the API endpoints. - - `name: string` + - `created_at: number` - - `sampled_model_name: string` + The Unix timestamp (in seconds) for when the permission was created. - - `scores: map[unknown]` + - `object: "checkpoint.permission"` - - `token_usage: number` + The object type, which is always "checkpoint.permission". - - `type: string` + - `"checkpoint.permission"` -- `model_grader_token_usage_per_model: map[unknown]` + - `project_id: string` -- `reward: number` + The project identifier that the permission is for. -- `sub_rewards: map[unknown]` +### Permission Create Response -### Example +- `PermissionCreateResponse object { id, created_at, object, project_id }` -```http -curl https://api.openai.com/v1/fine_tuning/alpha/graders/run \ - -H 'Content-Type: application/json' \ - -H "Authorization: Bearer $OPENAI_API_KEY" \ - -d '{ - "grader": { - "input": "input", - "name": "name", - "operation": "eq", - "reference": "reference", - "type": "string_check" - }, - "model_sample": "model_sample" - }' -``` + The `checkpoint.permission` object represents a permission for a fine-tuned model checkpoint. -#### Response + - `id: string` -```json -{ - "metadata": { - "errors": { - "formula_parse_error": true, - "invalid_variable_error": true, - "model_grader_parse_error": true, - "model_grader_refusal_error": true, - "model_grader_server_error": true, - "model_grader_server_error_details": "model_grader_server_error_details", - "other_error": true, - "python_grader_runtime_error": true, - "python_grader_runtime_error_details": "python_grader_runtime_error_details", - "python_grader_server_error": true, - "python_grader_server_error_type": "python_grader_server_error_type", - "sample_parse_error": true, - "truncated_observation_error": true, - "unresponsive_reward_error": true - }, - "execution_time": 0, - "name": "name", - "sampled_model_name": "sampled_model_name", - "scores": { - "foo": "bar" - }, - "token_usage": 0, - "type": "type" - }, - "model_grader_token_usage_per_model": { - "foo": "bar" - }, - "reward": 0, - "sub_rewards": { - "foo": "bar" - } -} -``` + The permission identifier, which can be referenced in the API endpoints. -### Score text alignment + - `created_at: number` -```http -curl -X POST https://api.openai.com/v1/fine_tuning/alpha/graders/run \ - -H "Content-Type: application/json" \ - -H "Authorization: Bearer $OPENAI_API_KEY" \ - -d '{ - "grader": { - "type": "score_model", - "name": "Example score model grader", - "input": [ - { - "role": "user", - "content": [ - { - "type": "input_text", - "text": "Score how close the reference answer is to the model answer on a 0-1 scale. Return only the score.\n\nReference answer: {{item.reference_answer}}\n\nModel answer: {{sample.output_text}}" - } - ] - } - ], - "model": "gpt-5-mini", - "sampling_params": { - "temperature": 1, - "top_p": 1, - "seed": 42 - } - }, - "item": { - "reference_answer": "fuzzy wuzzy was a bear" - }, - "model_sample": "fuzzy wuzzy was a bear" - }' -``` + The Unix timestamp (in seconds) for when the permission was created. -#### Response + - `object: "checkpoint.permission"` -```json -{ - "reward": 1.0, - "metadata": { - "name": "Example score model grader", - "type": "score_model", - "errors": { - "formula_parse_error": false, - "sample_parse_error": false, - "truncated_observation_error": false, - "unresponsive_reward_error": false, - "invalid_variable_error": false, - "other_error": false, - "python_grader_server_error": false, - "python_grader_server_error_type": null, - "python_grader_runtime_error": false, - "python_grader_runtime_error_details": null, - "model_grader_server_error": false, - "model_grader_refusal_error": false, - "model_grader_parse_error": false, - "model_grader_server_error_details": null - }, - "execution_time": 4.365238428115845, - "scores": {}, - "token_usage": { - "prompt_tokens": 190, - "total_tokens": 324, - "completion_tokens": 134, - "cached_tokens": 0 - }, - "sampled_model_name": "gpt-4o-2024-08-06" - }, - "sub_rewards": {}, - "model_grader_token_usage_per_model": { - "gpt-4o-2024-08-06": { - "prompt_tokens": 190, - "total_tokens": 324, - "completion_tokens": 134, - "cached_tokens": 0 - } - } -} -``` + The object type, which is always "checkpoint.permission". -### Score an image caption + - `"checkpoint.permission"` -```http -curl -X POST https://api.openai.com/v1/fine_tuning/alpha/graders/run \ - -H "Content-Type: application/json" \ - -H "Authorization: Bearer $OPENAI_API_KEY" \ - -d '{ - "grader": { - "type": "score_model", - "name": "Image caption grader", - "input": [ - { - "role": "user", - "content": [ - { - "type": "input_text", - "text": "Score how well the provided caption matches the image on a 0-1 scale. Only return the score.\n\nCaption: {{sample.output_text}}" - }, - { - "type": "input_image", - "image_url": "https://example.com/dog-catching-ball.png", - "file_id": null, - "detail": "high" - } - ] - } - ], - "model": "gpt-5-mini", - "sampling_params": { - "temperature": 0.2 - } - }, - "item": { - "expected_caption": "A golden retriever jumps to catch a tennis ball" - }, - "model_sample": "A dog leaps to grab a tennis ball mid-air" - }' -``` + - `project_id: string` -### Score an audio response + The project identifier that the permission is for. -```http -curl -X POST https://api.openai.com/v1/fine_tuning/alpha/graders/run \ - -H "Content-Type: application/json" \ - -H "Authorization: Bearer $OPENAI_API_KEY" \ - -d '{ - "grader": { - "type": "score_model", - "name": "Audio clarity grader", - "input": [ - { - "role": "user", - "content": [ - { - "type": "input_text", - "text": "Listen to the clip and return a confidence score from 0 to 1 that the speaker said: {{item.target_phrase}}" - }, - { - "type": "input_audio", - "input_audio": { - "data": "{{item.audio_clip_b64}}", - "format": "mp3" - } - } - ] - } - ], - "model": "gpt-audio", - "sampling_params": { - "temperature": 0.2, - "top_p": 1, - "seed": 123 - } - }, - "item": { - "target_phrase": "Please deliver the package on Tuesday", - "audio_clip_b64": "" - }, - "model_sample": "Please deliver the package on Tuesday" - }' -``` +### Permission Delete Response -## Validate grader +- `PermissionDeleteResponse object { id, deleted, object }` -**post** `/fine_tuning/alpha/graders/validate` + - `id: string` -Validate a grader. + The ID of the fine-tuned model checkpoint permission that was deleted. + + - `deleted: boolean` + + Whether the fine-tuned model checkpoint permission was successfully deleted. + + - `object: "checkpoint.permission"` + + The object type, which is always "checkpoint.permission". + + - `"checkpoint.permission"` + +# Alpha + +# Graders + +## Run grader + +**post** `/fine_tuning/alpha/graders/run` + +Run a grader. ### Body Parameters @@ -14887,7 +9469,7 @@ Validate a grader. The grader used for the fine-tuning job. - - `StringCheckGrader = object { input, name, operation, 2 more }` + - `StringCheckGrader object { input, name, operation, 2 more }` A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. @@ -14921,7 +9503,7 @@ Validate a grader. - `"string_check"` - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` A TextSimilarityGrader object which grades text based on similarity metrics. @@ -14965,41 +9547,286 @@ Validate a grader. The text being graded against. - - `type: "text_similarity"` + - `type: "text_similarity"` + + The type of grader. + + - `"text_similarity"` + + - `PythonGrader object { name, source, type, image_tag }` + + A PythonGrader object that runs a python script on the input. + + - `name: string` + + The name of the grader. + + - `source: string` + + The source code of the python script. + + - `type: "python"` + + The object type, which is always `python`. + + - `"python"` + + - `image_tag: optional string` + + The image tag to use for the python script. + + - `ScoreModelGrader object { input, model, name, 3 more }` + + A ScoreModelGrader object that uses a model to assign a score to the input. + + - `input: array of object { content, role, type }` + + The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. + + - `content: string or ResponseInputText or object { text, type } or 3 more` + + Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. + + - `TextInput = string` + + A text input to the model. + + - `ResponseInputText object { text, type }` + + A text input to the model. + + - `text: string` + + The text input to the model. + + - `type: "input_text"` + + The type of the input item. Always `input_text`. + + - `"input_text"` + + - `OutputText object { text, type }` + + A text output from the model. + + - `text: string` + + The text output from the model. + + - `type: "output_text"` + + The type of the output text. Always `output_text`. + + - `"output_text"` + + - `InputImage object { image_url, type, detail }` + + An image input block used within EvalItem content arrays. + + - `image_url: string` + + The URL of the image input. + + - `type: "input_image"` + + The type of the image input. Always `input_image`. + + - `"input_image"` + + - `detail: optional string` + + The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + + - `ResponseInputAudio object { input_audio, type }` + + An audio input to the model. + + - `input_audio: object { data, format }` + + - `data: string` + + Base64-encoded audio data. + + - `format: "mp3" or "wav"` + + The format of the audio data. Currently supported formats are `mp3` and + `wav`. + + - `"mp3"` + + - `"wav"` + + - `type: "input_audio"` + + The type of the input item. Always `input_audio`. + + - `"input_audio"` + + - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` + + A list of inputs, each of which may be either an input text, output text, input + image, or input audio object. + + - `TextInput = string` + + A text input to the model. + + - `ResponseInputText object { text, type }` + + A text input to the model. + + - `OutputText object { text, type }` + + A text output from the model. + + - `text: string` + + The text output from the model. + + - `type: "output_text"` + + The type of the output text. Always `output_text`. + + - `"output_text"` + + - `InputImage object { image_url, type, detail }` + + An image input block used within EvalItem content arrays. + + - `image_url: string` + + The URL of the image input. + + - `type: "input_image"` + + The type of the image input. Always `input_image`. + + - `"input_image"` + + - `detail: optional string` + + The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + + - `ResponseInputAudio object { input_audio, type }` + + An audio input to the model. + + - `role: "user" or "assistant" or "system" or "developer"` + + The role of the message input. One of `user`, `assistant`, `system`, or + `developer`. + + - `"user"` + + - `"assistant"` + + - `"system"` + + - `"developer"` + + - `type: optional "message"` + + The type of the message input. Always `message`. + + - `"message"` + + - `model: string` + + The model to use for the evaluation. + + - `name: string` + + The name of the grader. + + - `type: "score_model"` + + The object type, which is always `score_model`. + + - `"score_model"` + + - `range: optional array of number` + + The range of the score. Defaults to `[0, 1]`. + + - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` + + The sampling parameters for the model. + + - `max_completions_tokens: optional number` + + The maximum number of tokens the grader model may generate in its response. + + - `reasoning_effort: optional ReasoningEffort` + + Constrains effort on reasoning for + [reasoning models](https://platform.openai.com/docs/guides/reasoning). + Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing + reasoning effort can result in faster responses and fewer tokens used + on reasoning in a response. + + - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. + - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. + - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. + - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + + - `"none"` + + - `"minimal"` + + - `"low"` + + - `"medium"` + + - `"high"` + + - `"xhigh"` + + - `seed: optional number` + + A seed value to initialize the randomness, during sampling. + + - `temperature: optional number` + + A higher temperature increases randomness in the outputs. + + - `top_p: optional number` - The type of grader. + An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `"text_similarity"` + - `MultiGrader object { calculate_output, graders, name, type }` - - `PythonGrader = object { name, source, type, image_tag }` + A MultiGrader object combines the output of multiple graders to produce a single score. - A PythonGrader object that runs a python script on the input. + - `calculate_output: string` - - `name: string` + A formula to calculate the output based on grader results. - The name of the grader. + - `graders: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - - `source: string` + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - The source code of the python script. + - `StringCheckGrader object { input, name, operation, 2 more }` - - `type: "python"` + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - The object type, which is always `python`. + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` - - `"python"` + A TextSimilarityGrader object which grades text based on similarity metrics. - - `image_tag: optional string` + - `PythonGrader object { name, source, type, image_tag }` - The image tag to use for the python script. + A PythonGrader object that runs a python script on the input. - - `ScoreModelGrader = object { input, model, name, 3 more }` + - `ScoreModelGrader object { input, model, name, 3 more }` A ScoreModelGrader object that uses a model to assign a score to the input. - - `input: array of object { content, role, type }` + - `LabelModelGrader object { input, labels, model, 3 more }` - The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. + A LabelModelGrader object which uses a model to assign labels to each item + in the evaluation. + + - `input: array of object { content, role, type }` - `content: string or ResponseInputText or object { text, type } or 3 more` @@ -15009,21 +9836,11 @@ Validate a grader. A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -15037,7 +9854,7 @@ Validate a grader. - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -15055,206 +9872,364 @@ Validate a grader. The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` A list of inputs, each of which may be either an input text, output text, input image, or input audio object. - - `TextInput = string` - - A text input to the model. + - `role: "user" or "assistant" or "system" or "developer"` - - `ResponseInputText = object { text, type }` + The role of the message input. One of `user`, `assistant`, `system`, or + `developer`. - A text input to the model. + - `"user"` - - `text: string` + - `"assistant"` - The text input to the model. + - `"system"` - - `type: "input_text"` + - `"developer"` - The type of the input item. Always `input_text`. + - `type: optional "message"` - - `"input_text"` + The type of the message input. Always `message`. - - `OutputText = object { text, type }` + - `"message"` - A text output from the model. + - `labels: array of string` - - `text: string` + The labels to assign to each item in the evaluation. - The text output from the model. + - `model: string` - - `type: "output_text"` + The model to use for the evaluation. Must support structured outputs. - The type of the output text. Always `output_text`. + - `name: string` - - `"output_text"` + The name of the grader. - - `InputImage = object { image_url, type, detail }` + - `passing_labels: array of string` - An image input block used within EvalItem content arrays. + The labels that indicate a passing result. Must be a subset of labels. - - `image_url: string` + - `type: "label_model"` - The URL of the image input. + The object type, which is always `label_model`. - - `type: "input_image"` + - `"label_model"` - The type of the image input. Always `input_image`. + - `name: string` - - `"input_image"` + The name of the grader. - - `detail: optional string` + - `type: "multi"` - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + The object type, which is always `multi`. - - `ResponseInputAudio = object { input_audio, type }` + - `"multi"` - An audio input to the model. +- `model_sample: string` - - `input_audio: object { data, format }` + The model sample to be evaluated. This value will be used to populate + the `sample` namespace. See [the guide](/docs/guides/graders) for more details. + The `output_json` variable will be populated if the model sample is a + valid JSON string. - - `data: string` +- `item: optional unknown` - Base64-encoded audio data. + The dataset item provided to the grader. This will be used to populate + the `item` namespace. See [the guide](/docs/guides/graders) for more details. - - `format: "mp3" or "wav"` +### Returns - The format of the audio data. Currently supported formats are `mp3` and - `wav`. +- `metadata: object { errors, execution_time, name, 4 more }` - - `"mp3"` + - `errors: object { formula_parse_error, invalid_variable_error, model_grader_parse_error, 11 more }` - - `"wav"` + - `formula_parse_error: boolean` - - `type: "input_audio"` + - `invalid_variable_error: boolean` - The type of the input item. Always `input_audio`. + - `model_grader_parse_error: boolean` - - `"input_audio"` + - `model_grader_refusal_error: boolean` - - `role: "user" or "assistant" or "system" or "developer"` + - `model_grader_server_error: boolean` - The role of the message input. One of `user`, `assistant`, `system`, or - `developer`. + - `model_grader_server_error_details: string` - - `"user"` + - `other_error: boolean` - - `"assistant"` + - `python_grader_runtime_error: boolean` - - `"system"` + - `python_grader_runtime_error_details: string` - - `"developer"` + - `python_grader_server_error: boolean` - - `type: optional "message"` + - `python_grader_server_error_type: string` - The type of the message input. Always `message`. + - `sample_parse_error: boolean` - - `"message"` + - `truncated_observation_error: boolean` - - `model: string` + - `unresponsive_reward_error: boolean` - The model to use for the evaluation. + - `execution_time: number` - `name: string` - The name of the grader. - - - `type: "score_model"` - - The object type, which is always `score_model`. - - - `"score_model"` - - - `range: optional array of number` - - The range of the score. Defaults to `[0, 1]`. - - - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` + - `sampled_model_name: string` - The sampling parameters for the model. + - `scores: map[unknown]` - - `max_completions_tokens: optional number` + - `token_usage: number` - The maximum number of tokens the grader model may generate in its response. + - `type: string` - - `reasoning_effort: optional ReasoningEffort` +- `model_grader_token_usage_per_model: map[unknown]` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. +- `reward: number` - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. +- `sub_rewards: map[unknown]` - - `"none"` +### Example - - `"minimal"` +```http +curl https://api.openai.com/v1/fine_tuning/alpha/graders/run \ + -H 'Content-Type: application/json' \ + -H "Authorization: Bearer $OPENAI_API_KEY" \ + -d '{ + "grader": { + "input": "input", + "name": "name", + "operation": "eq", + "reference": "reference", + "type": "string_check" + }, + "model_sample": "model_sample" + }' +``` - - `"low"` +#### Response - - `"medium"` +```json +{ + "metadata": { + "errors": { + "formula_parse_error": true, + "invalid_variable_error": true, + "model_grader_parse_error": true, + "model_grader_refusal_error": true, + "model_grader_server_error": true, + "model_grader_server_error_details": "model_grader_server_error_details", + "other_error": true, + "python_grader_runtime_error": true, + "python_grader_runtime_error_details": "python_grader_runtime_error_details", + "python_grader_server_error": true, + "python_grader_server_error_type": "python_grader_server_error_type", + "sample_parse_error": true, + "truncated_observation_error": true, + "unresponsive_reward_error": true + }, + "execution_time": 0, + "name": "name", + "sampled_model_name": "sampled_model_name", + "scores": { + "foo": "bar" + }, + "token_usage": 0, + "type": "type" + }, + "model_grader_token_usage_per_model": { + "foo": "bar" + }, + "reward": 0, + "sub_rewards": { + "foo": "bar" + } +} +``` - - `"high"` +### Score text alignment - - `"xhigh"` +```http +curl -X POST https://api.openai.com/v1/fine_tuning/alpha/graders/run \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer $OPENAI_API_KEY" \ + -d '{ + "grader": { + "type": "score_model", + "name": "Example score model grader", + "input": [ + { + "role": "user", + "content": [ + { + "type": "input_text", + "text": "Score how close the reference answer is to the model answer on a 0-1 scale. Return only the score.\n\nReference answer: {{item.reference_answer}}\n\nModel answer: {{sample.output_text}}" + } + ] + } + ], + "model": "gpt-5-mini", + "sampling_params": { + "temperature": 1, + "top_p": 1, + "seed": 42 + } + }, + "item": { + "reference_answer": "fuzzy wuzzy was a bear" + }, + "model_sample": "fuzzy wuzzy was a bear" + }' +``` - - `seed: optional number` +#### Response - A seed value to initialize the randomness, during sampling. +```json +{ + "reward": 1.0, + "metadata": { + "name": "Example score model grader", + "type": "score_model", + "errors": { + "formula_parse_error": false, + "sample_parse_error": false, + "truncated_observation_error": false, + "unresponsive_reward_error": false, + "invalid_variable_error": false, + "other_error": false, + "python_grader_server_error": false, + "python_grader_server_error_type": null, + "python_grader_runtime_error": false, + "python_grader_runtime_error_details": null, + "model_grader_server_error": false, + "model_grader_refusal_error": false, + "model_grader_parse_error": false, + "model_grader_server_error_details": null + }, + "execution_time": 4.365238428115845, + "scores": {}, + "token_usage": { + "prompt_tokens": 190, + "total_tokens": 324, + "completion_tokens": 134, + "cached_tokens": 0 + }, + "sampled_model_name": "gpt-4o-2024-08-06" + }, + "sub_rewards": {}, + "model_grader_token_usage_per_model": { + "gpt-4o-2024-08-06": { + "prompt_tokens": 190, + "total_tokens": 324, + "completion_tokens": 134, + "cached_tokens": 0 + } + } +} +``` - - `temperature: optional number` +### Score an image caption - A higher temperature increases randomness in the outputs. +```http +curl -X POST https://api.openai.com/v1/fine_tuning/alpha/graders/run \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer $OPENAI_API_KEY" \ + -d '{ + "grader": { + "type": "score_model", + "name": "Image caption grader", + "input": [ + { + "role": "user", + "content": [ + { + "type": "input_text", + "text": "Score how well the provided caption matches the image on a 0-1 scale. Only return the score.\n\nCaption: {{sample.output_text}}" + }, + { + "type": "input_image", + "image_url": "https://example.com/dog-catching-ball.png", + "file_id": null, + "detail": "high" + } + ] + } + ], + "model": "gpt-5-mini", + "sampling_params": { + "temperature": 0.2 + } + }, + "item": { + "expected_caption": "A golden retriever jumps to catch a tennis ball" + }, + "model_sample": "A dog leaps to grab a tennis ball mid-air" + }' +``` - - `top_p: optional number` +### Score an audio response - An alternative to temperature for nucleus sampling; 1.0 includes all tokens. +```http +curl -X POST https://api.openai.com/v1/fine_tuning/alpha/graders/run \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer $OPENAI_API_KEY" \ + -d '{ + "grader": { + "type": "score_model", + "name": "Audio clarity grader", + "input": [ + { + "role": "user", + "content": [ + { + "type": "input_text", + "text": "Listen to the clip and return a confidence score from 0 to 1 that the speaker said: {{item.target_phrase}}" + }, + { + "type": "input_audio", + "input_audio": { + "data": "{{item.audio_clip_b64}}", + "format": "mp3" + } + } + ] + } + ], + "model": "gpt-audio", + "sampling_params": { + "temperature": 0.2, + "top_p": 1, + "seed": 123 + } + }, + "item": { + "target_phrase": "Please deliver the package on Tuesday", + "audio_clip_b64": "" + }, + "model_sample": "Please deliver the package on Tuesday" + }' +``` - - `MultiGrader = object { calculate_output, graders, name, type }` +## Validate grader - A MultiGrader object combines the output of multiple graders to produce a single score. +**post** `/fine_tuning/alpha/graders/validate` - - `calculate_output: string` +Validate a grader. - A formula to calculate the output based on grader results. +### Body Parameters - - `graders: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` +- `grader: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. + The grader used for the fine-tuning job. - - `StringCheckGrader = object { input, name, operation, 2 more }` + - `StringCheckGrader object { input, name, operation, 2 more }` A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. @@ -15288,7 +10263,7 @@ Validate a grader. - `"string_check"` - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` A TextSimilarityGrader object which grades text based on similarity metrics. @@ -15338,7 +10313,7 @@ Validate a grader. - `"text_similarity"` - - `PythonGrader = object { name, source, type, image_tag }` + - `PythonGrader object { name, source, type, image_tag }` A PythonGrader object that runs a python script on the input. @@ -15360,7 +10335,7 @@ Validate a grader. The image tag to use for the python script. - - `ScoreModelGrader = object { input, model, name, 3 more }` + - `ScoreModelGrader object { input, model, name, 3 more }` A ScoreModelGrader object that uses a model to assign a score to the input. @@ -15376,7 +10351,7 @@ Validate a grader. A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. @@ -15390,7 +10365,7 @@ Validate a grader. - `"input_text"` - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -15404,7 +10379,7 @@ Validate a grader. - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -15422,7 +10397,7 @@ Validate a grader. The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. @@ -15456,21 +10431,11 @@ Validate a grader. A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -15484,7 +10449,7 @@ Validate a grader. - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -15502,31 +10467,10 @@ Validate a grader. The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or @@ -15564,161 +10508,99 @@ Validate a grader. The range of the score. Defaults to `[0, 1]`. - - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` - - The sampling parameters for the model. - - - `max_completions_tokens: optional number` - - The maximum number of tokens the grader model may generate in its response. - - - `reasoning_effort: optional ReasoningEffort` - - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. - - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. - - - `"none"` - - - `"minimal"` - - - `"low"` - - - `"medium"` - - - `"high"` - - - `"xhigh"` - - - `seed: optional number` - - A seed value to initialize the randomness, during sampling. - - - `temperature: optional number` - - A higher temperature increases randomness in the outputs. - - - `top_p: optional number` - - An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - - `LabelModelGrader = object { input, labels, model, 3 more }` - - A LabelModelGrader object which uses a model to assign labels to each item - in the evaluation. - - - `input: array of object { content, role, type }` - - - `content: string or ResponseInputText or object { text, type } or 3 more` - - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - - - `TextInput = string` - - A text input to the model. + - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` - - `ResponseInputText = object { text, type }` + The sampling parameters for the model. - A text input to the model. + - `max_completions_tokens: optional number` - - `text: string` + The maximum number of tokens the grader model may generate in its response. - The text input to the model. + - `reasoning_effort: optional ReasoningEffort` - - `type: "input_text"` + Constrains effort on reasoning for + [reasoning models](https://platform.openai.com/docs/guides/reasoning). + Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing + reasoning effort can result in faster responses and fewer tokens used + on reasoning in a response. - The type of the input item. Always `input_text`. + - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. + - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. + - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. + - `xhigh` is supported for all models after `gpt-5.1-codex-max`. - - `"input_text"` + - `"none"` - - `OutputText = object { text, type }` + - `"minimal"` - A text output from the model. + - `"low"` - - `text: string` + - `"medium"` - The text output from the model. + - `"high"` - - `type: "output_text"` + - `"xhigh"` - The type of the output text. Always `output_text`. + - `seed: optional number` - - `"output_text"` + A seed value to initialize the randomness, during sampling. - - `InputImage = object { image_url, type, detail }` + - `temperature: optional number` - An image input block used within EvalItem content arrays. + A higher temperature increases randomness in the outputs. - - `image_url: string` + - `top_p: optional number` - The URL of the image input. + An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `type: "input_image"` + - `MultiGrader object { calculate_output, graders, name, type }` - The type of the image input. Always `input_image`. + A MultiGrader object combines the output of multiple graders to produce a single score. - - `"input_image"` + - `calculate_output: string` - - `detail: optional string` + A formula to calculate the output based on grader results. - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + - `graders: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - - `ResponseInputAudio = object { input_audio, type }` + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - An audio input to the model. + - `StringCheckGrader object { input, name, operation, 2 more }` - - `input_audio: object { data, format }` + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - `data: string` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` - Base64-encoded audio data. + A TextSimilarityGrader object which grades text based on similarity metrics. - - `format: "mp3" or "wav"` + - `PythonGrader object { name, source, type, image_tag }` - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + A PythonGrader object that runs a python script on the input. - - `"mp3"` + - `ScoreModelGrader object { input, model, name, 3 more }` - - `"wav"` + A ScoreModelGrader object that uses a model to assign a score to the input. - - `type: "input_audio"` + - `LabelModelGrader object { input, labels, model, 3 more }` - The type of the input item. Always `input_audio`. + A LabelModelGrader object which uses a model to assign labels to each item + in the evaluation. - - `"input_audio"` + - `input: array of object { content, role, type }` - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` + - `content: string or ResponseInputText or object { text, type } or 3 more` - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. + Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - `TextInput = string` A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -15732,7 +10614,7 @@ Validate a grader. - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -15750,30 +10632,14 @@ Validate a grader. The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. + - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - `"input_audio"` + A list of inputs, each of which may be either an input text, output text, input + image, or input audio object. - `role: "user" or "assistant" or "system" or "developer"` @@ -15832,7 +10698,7 @@ Validate a grader. The grader used for the fine-tuning job. - - `StringCheckGrader = object { input, name, operation, 2 more }` + - `StringCheckGrader object { input, name, operation, 2 more }` A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. @@ -15866,7 +10732,7 @@ Validate a grader. - `"string_check"` - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` A TextSimilarityGrader object which grades text based on similarity metrics. @@ -15902,153 +10768,308 @@ Validate a grader. The text being graded. - - `name: string` + - `name: string` + + The name of the grader. + + - `reference: string` + + The text being graded against. + + - `type: "text_similarity"` + + The type of grader. + + - `"text_similarity"` + + - `PythonGrader object { name, source, type, image_tag }` + + A PythonGrader object that runs a python script on the input. + + - `name: string` + + The name of the grader. + + - `source: string` + + The source code of the python script. + + - `type: "python"` + + The object type, which is always `python`. + + - `"python"` + + - `image_tag: optional string` + + The image tag to use for the python script. + + - `ScoreModelGrader object { input, model, name, 3 more }` + + A ScoreModelGrader object that uses a model to assign a score to the input. + + - `input: array of object { content, role, type }` + + The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. + + - `content: string or ResponseInputText or object { text, type } or 3 more` + + Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. + + - `TextInput = string` + + A text input to the model. + + - `ResponseInputText object { text, type }` + + A text input to the model. + + - `text: string` + + The text input to the model. + + - `type: "input_text"` + + The type of the input item. Always `input_text`. + + - `"input_text"` + + - `OutputText object { text, type }` + + A text output from the model. + + - `text: string` + + The text output from the model. + + - `type: "output_text"` + + The type of the output text. Always `output_text`. + + - `"output_text"` + + - `InputImage object { image_url, type, detail }` + + An image input block used within EvalItem content arrays. + + - `image_url: string` + + The URL of the image input. + + - `type: "input_image"` + + The type of the image input. Always `input_image`. + + - `"input_image"` + + - `detail: optional string` + + The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + + - `ResponseInputAudio object { input_audio, type }` + + An audio input to the model. + + - `input_audio: object { data, format }` + + - `data: string` + + Base64-encoded audio data. + + - `format: "mp3" or "wav"` + + The format of the audio data. Currently supported formats are `mp3` and + `wav`. + + - `"mp3"` + + - `"wav"` + + - `type: "input_audio"` + + The type of the input item. Always `input_audio`. + + - `"input_audio"` + + - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` + + A list of inputs, each of which may be either an input text, output text, input + image, or input audio object. + + - `TextInput = string` + + A text input to the model. + + - `ResponseInputText object { text, type }` + + A text input to the model. + + - `OutputText object { text, type }` + + A text output from the model. + + - `text: string` + + The text output from the model. + + - `type: "output_text"` + + The type of the output text. Always `output_text`. + + - `"output_text"` + + - `InputImage object { image_url, type, detail }` + + An image input block used within EvalItem content arrays. + + - `image_url: string` + + The URL of the image input. + + - `type: "input_image"` - The name of the grader. + The type of the image input. Always `input_image`. - - `reference: string` + - `"input_image"` - The text being graded against. + - `detail: optional string` - - `type: "text_similarity"` + The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - The type of grader. + - `ResponseInputAudio object { input_audio, type }` - - `"text_similarity"` + An audio input to the model. - - `PythonGrader = object { name, source, type, image_tag }` + - `role: "user" or "assistant" or "system" or "developer"` - A PythonGrader object that runs a python script on the input. + The role of the message input. One of `user`, `assistant`, `system`, or + `developer`. - - `name: string` + - `"user"` - The name of the grader. + - `"assistant"` - - `source: string` + - `"system"` - The source code of the python script. + - `"developer"` - - `type: "python"` + - `type: optional "message"` - The object type, which is always `python`. + The type of the message input. Always `message`. - - `"python"` + - `"message"` - - `image_tag: optional string` + - `model: string` - The image tag to use for the python script. + The model to use for the evaluation. - - `ScoreModelGrader = object { input, model, name, 3 more }` + - `name: string` - A ScoreModelGrader object that uses a model to assign a score to the input. + The name of the grader. - - `input: array of object { content, role, type }` + - `type: "score_model"` - The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. + The object type, which is always `score_model`. - - `content: string or ResponseInputText or object { text, type } or 3 more` + - `"score_model"` - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. + - `range: optional array of number` - - `TextInput = string` + The range of the score. Defaults to `[0, 1]`. - A text input to the model. + - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` - - `ResponseInputText = object { text, type }` + The sampling parameters for the model. - A text input to the model. + - `max_completions_tokens: optional number` - - `text: string` + The maximum number of tokens the grader model may generate in its response. - The text input to the model. + - `reasoning_effort: optional ReasoningEffort` - - `type: "input_text"` + Constrains effort on reasoning for + [reasoning models](https://platform.openai.com/docs/guides/reasoning). + Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing + reasoning effort can result in faster responses and fewer tokens used + on reasoning in a response. - The type of the input item. Always `input_text`. + - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. + - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. + - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. + - `xhigh` is supported for all models after `gpt-5.1-codex-max`. - - `"input_text"` + - `"none"` - - `OutputText = object { text, type }` + - `"minimal"` - A text output from the model. + - `"low"` - - `text: string` + - `"medium"` - The text output from the model. + - `"high"` - - `type: "output_text"` + - `"xhigh"` - The type of the output text. Always `output_text`. + - `seed: optional number` - - `"output_text"` + A seed value to initialize the randomness, during sampling. - - `InputImage = object { image_url, type, detail }` + - `temperature: optional number` - An image input block used within EvalItem content arrays. + A higher temperature increases randomness in the outputs. - - `image_url: string` + - `top_p: optional number` - The URL of the image input. + An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `type: "input_image"` + - `MultiGrader object { calculate_output, graders, name, type }` - The type of the image input. Always `input_image`. + A MultiGrader object combines the output of multiple graders to produce a single score. - - `"input_image"` + - `calculate_output: string` - - `detail: optional string` + A formula to calculate the output based on grader results. - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + - `graders: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - - `ResponseInputAudio = object { input_audio, type }` + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - An audio input to the model. + - `StringCheckGrader object { input, name, operation, 2 more }` - - `input_audio: object { data, format }` + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - `data: string` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` - Base64-encoded audio data. + A TextSimilarityGrader object which grades text based on similarity metrics. - - `format: "mp3" or "wav"` + - `PythonGrader object { name, source, type, image_tag }` - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + A PythonGrader object that runs a python script on the input. - - `"mp3"` + - `ScoreModelGrader object { input, model, name, 3 more }` - - `"wav"` + A ScoreModelGrader object that uses a model to assign a score to the input. - - `type: "input_audio"` + - `LabelModelGrader object { input, labels, model, 3 more }` - The type of the input item. Always `input_audio`. + A LabelModelGrader object which uses a model to assign labels to each item + in the evaluation. - - `"input_audio"` + - `input: array of object { content, role, type }` - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` + - `content: string or ResponseInputText or object { text, type } or 3 more` - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. + Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - `TextInput = string` A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -16062,7 +11083,7 @@ Validate a grader. - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -16080,30 +11101,14 @@ Validate a grader. The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. + - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - `"input_audio"` + A list of inputs, each of which may be either an input text, output text, input + image, or input audio object. - `role: "user" or "assistant" or "system" or "developer"` @@ -16124,82 +11129,165 @@ Validate a grader. - `"message"` + - `labels: array of string` + + The labels to assign to each item in the evaluation. + - `model: string` - The model to use for the evaluation. + The model to use for the evaluation. Must support structured outputs. - `name: string` The name of the grader. - - `type: "score_model"` + - `passing_labels: array of string` - The object type, which is always `score_model`. + The labels that indicate a passing result. Must be a subset of labels. - - `"score_model"` + - `type: "label_model"` - - `range: optional array of number` + The object type, which is always `label_model`. - The range of the score. Defaults to `[0, 1]`. + - `"label_model"` + + - `name: string` + + The name of the grader. + + - `type: "multi"` + + The object type, which is always `multi`. + + - `"multi"` + +### Example + +```http +curl https://api.openai.com/v1/fine_tuning/alpha/graders/validate \ + -H 'Content-Type: application/json' \ + -H "Authorization: Bearer $OPENAI_API_KEY" \ + -d '{ + "grader": { + "input": "input", + "name": "name", + "operation": "eq", + "reference": "reference", + "type": "string_check" + } + }' +``` + +#### Response + +```json +{ + "grader": { + "input": "input", + "name": "name", + "operation": "eq", + "reference": "reference", + "type": "string_check" + } +} +``` + +### Example + +```http +curl https://api.openai.com/v1/fine_tuning/alpha/graders/validate \ + -H "Authorization: Bearer $OPENAI_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{ + "grader": { + "type": "string_check", + "name": "Example string check grader", + "input": "{{sample.output_text}}", + "reference": "{{item.label}}", + "operation": "eq" + } + }' +``` + +#### Response + +```json +{ + "grader": { + "type": "string_check", + "name": "Example string check grader", + "input": "{{sample.output_text}}", + "reference": "{{item.label}}", + "operation": "eq" + } +} +``` + +## Domain Types + +### Grader Run Response + +- `GraderRunResponse object { metadata, model_grader_token_usage_per_model, reward, sub_rewards }` + + - `metadata: object { errors, execution_time, name, 4 more }` + + - `errors: object { formula_parse_error, invalid_variable_error, model_grader_parse_error, 11 more }` + + - `formula_parse_error: boolean` + + - `invalid_variable_error: boolean` - - `sampling_params: optional object { max_completions_tokens, reasoning_effort, seed, 2 more }` + - `model_grader_parse_error: boolean` - The sampling parameters for the model. + - `model_grader_refusal_error: boolean` - - `max_completions_tokens: optional number` + - `model_grader_server_error: boolean` - The maximum number of tokens the grader model may generate in its response. + - `model_grader_server_error_details: string` - - `reasoning_effort: optional ReasoningEffort` + - `other_error: boolean` - Constrains effort on reasoning for - [reasoning models](https://platform.openai.com/docs/guides/reasoning). - Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing - reasoning effort can result in faster responses and fewer tokens used - on reasoning in a response. + - `python_grader_runtime_error: boolean` - - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - - `xhigh` is supported for all models after `gpt-5.1-codex-max`. + - `python_grader_runtime_error_details: string` - - `"none"` + - `python_grader_server_error: boolean` - - `"minimal"` + - `python_grader_server_error_type: string` - - `"low"` + - `sample_parse_error: boolean` - - `"medium"` + - `truncated_observation_error: boolean` - - `"high"` + - `unresponsive_reward_error: boolean` - - `"xhigh"` + - `execution_time: number` - - `seed: optional number` + - `name: string` - A seed value to initialize the randomness, during sampling. + - `sampled_model_name: string` - - `temperature: optional number` + - `scores: map[unknown]` - A higher temperature increases randomness in the outputs. + - `token_usage: number` - - `top_p: optional number` + - `type: string` - An alternative to temperature for nucleus sampling; 1.0 includes all tokens. + - `model_grader_token_usage_per_model: map[unknown]` - - `MultiGrader = object { calculate_output, graders, name, type }` + - `reward: number` - A MultiGrader object combines the output of multiple graders to produce a single score. + - `sub_rewards: map[unknown]` - - `calculate_output: string` +### Grader Validate Response - A formula to calculate the output based on grader results. +- `GraderValidateResponse object { grader }` - - `graders: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` + - `grader: optional StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. + The grader used for the fine-tuning job. - - `StringCheckGrader = object { input, name, operation, 2 more }` + - `StringCheckGrader object { input, name, operation, 2 more }` A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. @@ -16233,7 +11321,7 @@ Validate a grader. - `"string_check"` - - `TextSimilarityGrader = object { evaluation_metric, input, name, 2 more }` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` A TextSimilarityGrader object which grades text based on similarity metrics. @@ -16283,7 +11371,7 @@ Validate a grader. - `"text_similarity"` - - `PythonGrader = object { name, source, type, image_tag }` + - `PythonGrader object { name, source, type, image_tag }` A PythonGrader object that runs a python script on the input. @@ -16305,7 +11393,7 @@ Validate a grader. The image tag to use for the python script. - - `ScoreModelGrader = object { input, model, name, 3 more }` + - `ScoreModelGrader object { input, model, name, 3 more }` A ScoreModelGrader object that uses a model to assign a score to the input. @@ -16321,7 +11409,7 @@ Validate a grader. A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. @@ -16335,7 +11423,7 @@ Validate a grader. - `"input_text"` - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -16349,7 +11437,7 @@ Validate a grader. - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -16367,7 +11455,7 @@ Validate a grader. The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. @@ -16401,21 +11489,11 @@ Validate a grader. A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -16429,7 +11507,7 @@ Validate a grader. - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -16447,31 +11525,10 @@ Validate a grader. The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. - - - `"input_audio"` - - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or @@ -16554,116 +11611,54 @@ Validate a grader. An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - - `LabelModelGrader = object { input, labels, model, 3 more }` - - A LabelModelGrader object which uses a model to assign labels to each item - in the evaluation. - - - `input: array of object { content, role, type }` - - - `content: string or ResponseInputText or object { text, type } or 3 more` - - Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - - - `TextInput = string` - - A text input to the model. - - - `ResponseInputText = object { text, type }` - - A text input to the model. - - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` - - A text output from the model. - - - `text: string` - - The text output from the model. - - - `type: "output_text"` - - The type of the output text. Always `output_text`. - - - `"output_text"` - - - `InputImage = object { image_url, type, detail }` - - An image input block used within EvalItem content arrays. - - - `image_url: string` - - The URL of the image input. - - - `type: "input_image"` + - `MultiGrader object { calculate_output, graders, name, type }` - The type of the image input. Always `input_image`. + A MultiGrader object combines the output of multiple graders to produce a single score. - - `"input_image"` + - `calculate_output: string` - - `detail: optional string` + A formula to calculate the output based on grader results. - The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. + - `graders: StringCheckGrader or TextSimilarityGrader or PythonGrader or 2 more` - - `ResponseInputAudio = object { input_audio, type }` + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - An audio input to the model. + - `StringCheckGrader object { input, name, operation, 2 more }` - - `input_audio: object { data, format }` + A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - - `data: string` + - `TextSimilarityGrader object { evaluation_metric, input, name, 2 more }` - Base64-encoded audio data. + A TextSimilarityGrader object which grades text based on similarity metrics. - - `format: "mp3" or "wav"` + - `PythonGrader object { name, source, type, image_tag }` - The format of the audio data. Currently supported formats are `mp3` and - `wav`. + A PythonGrader object that runs a python script on the input. - - `"mp3"` + - `ScoreModelGrader object { input, model, name, 3 more }` - - `"wav"` + A ScoreModelGrader object that uses a model to assign a score to the input. - - `type: "input_audio"` + - `LabelModelGrader object { input, labels, model, 3 more }` - The type of the input item. Always `input_audio`. + A LabelModelGrader object which uses a model to assign labels to each item + in the evaluation. - - `"input_audio"` + - `input: array of object { content, role, type }` - - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` + - `content: string or ResponseInputText or object { text, type } or 3 more` - A list of inputs, each of which may be either an input text, output text, input - image, or input audio object. + Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - `TextInput = string` A text input to the model. - - `ResponseInputText = object { text, type }` + - `ResponseInputText object { text, type }` A text input to the model. - - `text: string` - - The text input to the model. - - - `type: "input_text"` - - The type of the input item. Always `input_text`. - - - `"input_text"` - - - `OutputText = object { text, type }` + - `OutputText object { text, type }` A text output from the model. @@ -16677,7 +11672,7 @@ Validate a grader. - `"output_text"` - - `InputImage = object { image_url, type, detail }` + - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. @@ -16695,30 +11690,14 @@ Validate a grader. The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - - `ResponseInputAudio = object { input_audio, type }` + - `ResponseInputAudio object { input_audio, type }` An audio input to the model. - - `input_audio: object { data, format }` - - - `data: string` - - Base64-encoded audio data. - - - `format: "mp3" or "wav"` - - The format of the audio data. Currently supported formats are `mp3` and - `wav`. - - - `"mp3"` - - - `"wav"` - - - `type: "input_audio"` - - The type of the input item. Always `input_audio`. + - `GraderInputs = array of string or ResponseInputText or object { text, type } or 2 more` - - `"input_audio"` + A list of inputs, each of which may be either an input text, output text, input + image, or input audio object. - `role: "user" or "assistant" or "system" or "developer"` @@ -16770,65 +11749,3 @@ Validate a grader. The object type, which is always `multi`. - `"multi"` - -### Example - -```http -curl https://api.openai.com/v1/fine_tuning/alpha/graders/validate \ - -H 'Content-Type: application/json' \ - -H "Authorization: Bearer $OPENAI_API_KEY" \ - -d '{ - "grader": { - "input": "input", - "name": "name", - "operation": "eq", - "reference": "reference", - "type": "string_check" - } - }' -``` - -#### Response - -```json -{ - "grader": { - "input": "input", - "name": "name", - "operation": "eq", - "reference": "reference", - "type": "string_check" - } -} -``` - -### Example - -```http -curl https://api.openai.com/v1/fine_tuning/alpha/graders/validate \ - -H "Authorization: Bearer $OPENAI_API_KEY" \ - -H "Content-Type: application/json" \ - -d '{ - "grader": { - "type": "string_check", - "name": "Example string check grader", - "input": "{{sample.output_text}}", - "reference": "{{item.label}}", - "operation": "eq" - } - }' -``` - -#### Response - -```json -{ - "grader": { - "type": "string_check", - "name": "Example string check grader", - "input": "{{sample.output_text}}", - "reference": "{{item.label}}", - "operation": "eq" - } -} -```