python/resources/fine_tuning/subresources/alpha/index.md 2026-05-18 22:01 UTC to 2026-05-19 06:34 UTC

1# Alpha

2 

3# Graders

4 

5## Run grader

6 

7`fine_tuning.alpha.graders.run(GraderRunParams**kwargs) -> GraderRunResponse`

8 

9**post** `/fine_tuning/alpha/graders/run`

10 

11Run a grader.

12 

13### Parameters

14 

15- `grader: Grader`

16 

17 The grader used for the fine-tuning job.

18 

19 - `class StringCheckGrader: …`

20 

21 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.

22 

23 - `input: str`

24 

25 The input text. This may include template strings.

26 

27 - `name: str`

28 

29 The name of the grader.

30 

31 - `operation: Literal["eq", "ne", "like", "ilike"]`

32 

33 The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.

34 

35 - `"eq"`

36 

37 - `"ne"`

38 

39 - `"like"`

40 

41 - `"ilike"`

42 

43 - `reference: str`

44 

45 The reference text. This may include template strings.

46 

47 - `type: Literal["string_check"]`

48 

49 The object type, which is always `string_check`.

50 

51 - `"string_check"`

52 

53 - `class TextSimilarityGrader: …`

54 

55 A TextSimilarityGrader object which grades text based on similarity metrics.

56 

57 - `evaluation_metric: Literal["cosine", "fuzzy_match", "bleu", 8 more]`

58 

59 The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`,

60 `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`,

61 or `rouge_l`.

62 

63 - `"cosine"`

64 

65 - `"fuzzy_match"`

66 

67 - `"bleu"`

68 

69 - `"gleu"`

70 

71 - `"meteor"`

72 

73 - `"rouge_1"`

74 

75 - `"rouge_2"`

76 

77 - `"rouge_3"`

78 

79 - `"rouge_4"`

80 

81 - `"rouge_5"`

82 

83 - `"rouge_l"`

84 

85 - `input: str`

86 

87 The text being graded.

88 

89 - `name: str`

90 

91 The name of the grader.

92 

93 - `reference: str`

94 

95 The text being graded against.

96 

97 - `type: Literal["text_similarity"]`

98 

99 The type of grader.

100 

101 - `"text_similarity"`

102 

103 - `class PythonGrader: …`

104 

105 A PythonGrader object that runs a python script on the input.

106 

107 - `name: str`

108 

109 The name of the grader.

110 

111 - `source: str`

112 

113 The source code of the python script.

114 

115 - `type: Literal["python"]`

116 

117 The object type, which is always `python`.

118 

119 - `"python"`

120 

121 - `image_tag: Optional[str]`

122 

123 The image tag to use for the python script.

124 

125 - `class ScoreModelGrader: …`

126 

127 A ScoreModelGrader object that uses a model to assign a score to the input.

128 

129 - `input: List[Input]`

130 

131 The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.

132 

133 - `content: InputContent`

134 

135 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

136 

137 - `str`

138 

139 A text input to the model.

140 

141 - `class ResponseInputText: …`

142 

143 A text input to the model.

144 

145 - `text: str`

146 

147 The text input to the model.

148 

149 - `type: Literal["input_text"]`

150 

151 The type of the input item. Always `input_text`.

152 

153 - `"input_text"`

154 

155 - `class InputContentOutputText: …`

156 

157 A text output from the model.

158 

159 - `text: str`

160 

161 The text output from the model.

162 

163 - `type: Literal["output_text"]`

164 

165 The type of the output text. Always `output_text`.

166 

167 - `"output_text"`

168 

169 - `class InputContentInputImage: …`

170 

171 An image input block used within EvalItem content arrays.

172 

173 - `image_url: str`

174 

175 The URL of the image input.

176 

177 - `type: Literal["input_image"]`

178 

179 The type of the image input. Always `input_image`.

180 

181 - `"input_image"`

182 

183 - `detail: Optional[str]`

184 

185 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

186 

187 - `class ResponseInputAudio: …`

188 

189 An audio input to the model.

190 

191 - `input_audio: InputAudio`

192 

193 - `data: str`

194 

195 Base64-encoded audio data.

196 

197 - `format: Literal["mp3", "wav"]`

198 

199 The format of the audio data. Currently supported formats are `mp3` and

200 `wav`.

201 

202 - `"mp3"`

203 

204 - `"wav"`

205 

206 - `type: Literal["input_audio"]`

207 

208 The type of the input item. Always `input_audio`.

209 

210 - `"input_audio"`

211 

212 - `List[GraderInputItem]`

213 

214 - `str`

215 

216 A text input to the model.

217 

218 - `class ResponseInputText: …`

219 

220 A text input to the model.

221 

222 - `class GraderInputItemOutputText: …`

223 

224 A text output from the model.

225 

226 - `text: str`

227 

228 The text output from the model.

229 

230 - `type: Literal["output_text"]`

231 

232 The type of the output text. Always `output_text`.

233 

234 - `"output_text"`

235 

236 - `class GraderInputItemInputImage: …`

237 

238 An image input block used within EvalItem content arrays.

239 

240 - `image_url: str`

241 

242 The URL of the image input.

243 

244 - `type: Literal["input_image"]`

245 

246 The type of the image input. Always `input_image`.

247 

248 - `"input_image"`

249 

250 - `detail: Optional[str]`

251 

252 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

253 

254 - `class ResponseInputAudio: …`

255 

256 An audio input to the model.

257 

258 - `role: Literal["user", "assistant", "system", "developer"]`

259 

260 The role of the message input. One of `user`, `assistant`, `system`, or

261 `developer`.

262 

263 - `"user"`

264 

265 - `"assistant"`

266 

267 - `"system"`

268 

269 - `"developer"`

270 

271 - `type: Optional[Literal["message"]]`

272 

273 The type of the message input. Always `message`.

274 

275 - `"message"`

276 

277 - `model: str`

278 

279 The model to use for the evaluation.

280 

281 - `name: str`

282 

283 The name of the grader.

284 

285 - `type: Literal["score_model"]`

286 

287 The object type, which is always `score_model`.

288 

289 - `"score_model"`

290 

291 - `range: Optional[List[float]]`

292 

293 The range of the score. Defaults to `[0, 1]`.

294 

295 - `sampling_params: Optional[SamplingParams]`

296 

297 The sampling parameters for the model.

298 

299 - `max_completions_tokens: Optional[int]`

300 

301 The maximum number of tokens the grader model may generate in its response.

302 

303 - `reasoning_effort: Optional[ReasoningEffort]`

304 

305 Constrains effort on reasoning for

306 [reasoning models](https://platform.openai.com/docs/guides/reasoning).

307 Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing

308 reasoning effort can result in faster responses and fewer tokens used

309 on reasoning in a response.

310 

311 - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1.

312 - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`.

313 - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort.

314 - `xhigh` is supported for all models after `gpt-5.1-codex-max`.

315 

316 - `"none"`

317 

318 - `"minimal"`

319 

320 - `"low"`

321 

322 - `"medium"`

323 

324 - `"high"`

325 

326 - `"xhigh"`

327 

328 - `seed: Optional[int]`

329 

330 A seed value to initialize the randomness, during sampling.

331 

332 - `temperature: Optional[float]`

333 

334 A higher temperature increases randomness in the outputs.

335 

336 - `top_p: Optional[float]`

337 

338 An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

339 

340 - `class MultiGrader: …`

341 

342 A MultiGrader object combines the output of multiple graders to produce a single score.

343 

344 - `calculate_output: str`

345 

346 A formula to calculate the output based on grader results.

347 

348 - `graders: Graders`

349 

350 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.

351 

352 - `class StringCheckGrader: …`

353 

354 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.

355 

356 - `class TextSimilarityGrader: …`

357 

358 A TextSimilarityGrader object which grades text based on similarity metrics.

359 

360 - `class PythonGrader: …`

361 

362 A PythonGrader object that runs a python script on the input.

363 

364 - `class ScoreModelGrader: …`

365 

366 A ScoreModelGrader object that uses a model to assign a score to the input.

367 

368 - `class LabelModelGrader: …`

369 

370 A LabelModelGrader object which uses a model to assign labels to each item

371 in the evaluation.

372 

373 - `input: List[Input]`

374 

375 - `content: InputContent`

376 

377 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

378 

379 - `str`

380 

381 A text input to the model.

382 

383 - `class ResponseInputText: …`

384 

385 A text input to the model.

386 

387 - `class InputContentOutputText: …`

388 

389 A text output from the model.

390 

391 - `text: str`

392 

393 The text output from the model.

394 

395 - `type: Literal["output_text"]`

396 

397 The type of the output text. Always `output_text`.

398 

399 - `"output_text"`

400 

401 - `class InputContentInputImage: …`

402 

403 An image input block used within EvalItem content arrays.

404 

405 - `image_url: str`

406 

407 The URL of the image input.

408 

409 - `type: Literal["input_image"]`

410 

411 The type of the image input. Always `input_image`.

412 

413 - `"input_image"`

414 

415 - `detail: Optional[str]`

416 

417 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

418 

419 - `class ResponseInputAudio: …`

420 

421 An audio input to the model.

422 

423 - `List[GraderInputItem]`

424 

425 - `str`

426 

427 A text input to the model.

428 

429 - `class ResponseInputText: …`

430 

431 A text input to the model.

432 

433 - `class GraderInputItemOutputText: …`

434 

435 A text output from the model.

436 

437 - `class GraderInputItemInputImage: …`

438 

439 An image input block used within EvalItem content arrays.

440 

441 - `class ResponseInputAudio: …`

442 

443 An audio input to the model.

444 

445 - `role: Literal["user", "assistant", "system", "developer"]`

446 

447 The role of the message input. One of `user`, `assistant`, `system`, or

448 `developer`.

449 

450 - `"user"`

451 

452 - `"assistant"`

453 

454 - `"system"`

455 

456 - `"developer"`

457 

458 - `type: Optional[Literal["message"]]`

459 

460 The type of the message input. Always `message`.

461 

462 - `"message"`

463 

464 - `labels: List[str]`

465 

466 The labels to assign to each item in the evaluation.

467 

468 - `model: str`

469 

470 The model to use for the evaluation. Must support structured outputs.

471 

472 - `name: str`

473 

474 The name of the grader.

475 

476 - `passing_labels: List[str]`

477 

478 The labels that indicate a passing result. Must be a subset of labels.

479 

480 - `type: Literal["label_model"]`

481 

482 The object type, which is always `label_model`.

483 

484 - `"label_model"`

485 

486 - `name: str`

487 

488 The name of the grader.

489 

490 - `type: Literal["multi"]`

491 

492 The object type, which is always `multi`.

493 

494 - `"multi"`

495 

496- `model_sample: str`

497 

498 The model sample to be evaluated. This value will be used to populate

499 the `sample` namespace. See [the guide](https://platform.openai.com/docs/guides/graders) for more details.

500 The `output_json` variable will be populated if the model sample is a

501 valid JSON string.

502 

503- `item: Optional[object]`

504 

505 The dataset item provided to the grader. This will be used to populate

506 the `item` namespace. See [the guide](https://platform.openai.com/docs/guides/graders) for more details.

507 

508### Returns

509 

510- `class GraderRunResponse: …`

511 

512 - `metadata: Metadata`

513 

514 - `errors: MetadataErrors`

515 

516 - `formula_parse_error: bool`

517 

518 - `invalid_variable_error: bool`

519 

520 - `model_grader_parse_error: bool`

521 

522 - `model_grader_refusal_error: bool`

523 

524 - `model_grader_server_error: bool`

525 

526 - `model_grader_server_error_details: Optional[str]`

527 

528 - `other_error: bool`

529 

530 - `python_grader_runtime_error: bool`

531 

532 - `python_grader_runtime_error_details: Optional[str]`

533 

534 - `python_grader_server_error: bool`

535 

536 - `python_grader_server_error_type: Optional[str]`

537 

538 - `sample_parse_error: bool`

539 

540 - `truncated_observation_error: bool`

541 

542 - `unresponsive_reward_error: bool`

543 

544 - `execution_time: float`

545 

546 - `name: str`

547 

548 - `sampled_model_name: Optional[str]`

549 

550 - `scores: Dict[str, object]`

551 

552 - `token_usage: Optional[int]`

553 

554 - `type: str`

555 

556 - `model_grader_token_usage_per_model: Dict[str, object]`

557 

558 - `reward: float`

559 

560 - `sub_rewards: Dict[str, object]`

561 

562### Example

563 

564```python

565import os

566from openai import OpenAI

567 

568client = OpenAI(

569 api_key=os.environ.get("OPENAI_API_KEY"), # This is the default and can be omitted

570)

571response = client.fine_tuning.alpha.graders.run(

572 grader={

573 "input": "input",

574 "name": "name",

575 "operation": "eq",

576 "reference": "reference",

577 "type": "string_check",

578 },

579 model_sample="model_sample",

580)

581print(response.metadata)

582```

583 

584#### Response

585 

586```json

587{

588 "metadata": {

589 "errors": {

590 "formula_parse_error": true,

591 "invalid_variable_error": true,

592 "model_grader_parse_error": true,

593 "model_grader_refusal_error": true,

594 "model_grader_server_error": true,

595 "model_grader_server_error_details": "model_grader_server_error_details",

596 "other_error": true,

597 "python_grader_runtime_error": true,

598 "python_grader_runtime_error_details": "python_grader_runtime_error_details",

599 "python_grader_server_error": true,

600 "python_grader_server_error_type": "python_grader_server_error_type",

601 "sample_parse_error": true,

602 "truncated_observation_error": true,

603 "unresponsive_reward_error": true

604 },

605 "execution_time": 0,

606 "name": "name",

607 "sampled_model_name": "sampled_model_name",

608 "scores": {

609 "foo": "bar"

610 },

611 "token_usage": 0,

612 "type": "type"

613 },

614 "model_grader_token_usage_per_model": {

615 "foo": "bar"

616 },

617 "reward": 0,

618 "sub_rewards": {

619 "foo": "bar"

620 }

621}

622```

623 

624### Score text alignment

625 

626```python

627from openai import OpenAI

628 

629client = OpenAI()

630result = client.fine_tuning.alpha.graders.run(

631 grader={

632 "type": "score_model",

633 "name": "Example score model grader",

634 "input": [

635 {

636 "role": "user",

637 "content": [

638 {

639 "type": "input_text",

640 "text": "Score how close the reference answer is to the model answer on a 0-1 scale. Return only the score.\n\nReference answer: {{item.reference_answer}}\n\nModel answer: {{sample.output_text}}",

641 }

642 ],

643 }

644 ],

645 "model": "gpt-5-mini",

646 "sampling_params": {"temperature": 1, "top_p": 1, "seed": 42},

647 },

648 item={"reference_answer": "fuzzy wuzzy was a bear"},

649 model_sample="fuzzy wuzzy was a bear",

650)

651print(result)

652```

653 

654#### Response

655 

656```json

657{

658 "reward": 1.0,

659 "metadata": {

660 "name": "Example score model grader",

661 "type": "score_model",

662 "errors": {

663 "formula_parse_error": false,

664 "sample_parse_error": false,

665 "truncated_observation_error": false,

666 "unresponsive_reward_error": false,

667 "invalid_variable_error": false,

668 "other_error": false,

669 "python_grader_server_error": false,

670 "python_grader_server_error_type": null,

671 "python_grader_runtime_error": false,

672 "python_grader_runtime_error_details": null,

673 "model_grader_server_error": false,

674 "model_grader_refusal_error": false,

675 "model_grader_parse_error": false,

676 "model_grader_server_error_details": null

677 },

678 "execution_time": 4.365238428115845,

679 "scores": {},

680 "token_usage": {

681 "prompt_tokens": 190,

682 "total_tokens": 324,

683 "completion_tokens": 134,

684 "cached_tokens": 0

685 },

686 "sampled_model_name": "gpt-4o-2024-08-06"

687 },

688 "sub_rewards": {},

689 "model_grader_token_usage_per_model": {

690 "gpt-4o-2024-08-06": {

691 "prompt_tokens": 190,

692 "total_tokens": 324,

693 "completion_tokens": 134,

694 "cached_tokens": 0

695 }

696 }

697}

698```

699 

700## Validate grader

701 

702`fine_tuning.alpha.graders.validate(GraderValidateParams**kwargs) -> GraderValidateResponse`

703 

704**post** `/fine_tuning/alpha/graders/validate`

705 

706Validate a grader.

707 

708### Parameters

709 

710- `grader: Grader`

711 

712 The grader used for the fine-tuning job.

713 

714 - `class StringCheckGrader: …`

715 

716 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.

717 

718 - `input: str`

719 

720 The input text. This may include template strings.

721 

722 - `name: str`

723 

724 The name of the grader.

725 

726 - `operation: Literal["eq", "ne", "like", "ilike"]`

727 

728 The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.

729 

730 - `"eq"`

731 

732 - `"ne"`

733 

734 - `"like"`

735 

736 - `"ilike"`

737 

738 - `reference: str`

739 

740 The reference text. This may include template strings.

741 

742 - `type: Literal["string_check"]`

743 

744 The object type, which is always `string_check`.

745 

746 - `"string_check"`

747 

748 - `class TextSimilarityGrader: …`

749 

750 A TextSimilarityGrader object which grades text based on similarity metrics.

751 

752 - `evaluation_metric: Literal["cosine", "fuzzy_match", "bleu", 8 more]`

753 

754 The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`,

755 `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`,

756 or `rouge_l`.

757 

758 - `"cosine"`

759 

760 - `"fuzzy_match"`

761 

762 - `"bleu"`

763 

764 - `"gleu"`

765 

766 - `"meteor"`

767 

768 - `"rouge_1"`

769 

770 - `"rouge_2"`

771 

772 - `"rouge_3"`

773 

774 - `"rouge_4"`

775 

776 - `"rouge_5"`

777 

778 - `"rouge_l"`

779 

780 - `input: str`

781 

782 The text being graded.

783 

784 - `name: str`

785 

786 The name of the grader.

787 

788 - `reference: str`

789 

790 The text being graded against.

791 

792 - `type: Literal["text_similarity"]`

793 

794 The type of grader.

795 

796 - `"text_similarity"`

797 

798 - `class PythonGrader: …`

799 

800 A PythonGrader object that runs a python script on the input.

801 

802 - `name: str`

803 

804 The name of the grader.

805 

806 - `source: str`

807 

808 The source code of the python script.

809 

810 - `type: Literal["python"]`

811 

812 The object type, which is always `python`.

813 

814 - `"python"`

815 

816 - `image_tag: Optional[str]`

817 

818 The image tag to use for the python script.

819 

820 - `class ScoreModelGrader: …`

821 

822 A ScoreModelGrader object that uses a model to assign a score to the input.

823 

824 - `input: List[Input]`

825 

826 The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.

827 

828 - `content: InputContent`

829 

830 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

831 

832 - `str`

833 

834 A text input to the model.

835 

836 - `class ResponseInputText: …`

837 

838 A text input to the model.

839 

840 - `text: str`

841 

842 The text input to the model.

843 

844 - `type: Literal["input_text"]`

845 

846 The type of the input item. Always `input_text`.

847 

848 - `"input_text"`

849 

850 - `class InputContentOutputText: …`

851 

852 A text output from the model.

853 

854 - `text: str`

855 

856 The text output from the model.

857 

858 - `type: Literal["output_text"]`

859 

860 The type of the output text. Always `output_text`.

861 

862 - `"output_text"`

863 

864 - `class InputContentInputImage: …`

865 

866 An image input block used within EvalItem content arrays.

867 

868 - `image_url: str`

869 

870 The URL of the image input.

871 

872 - `type: Literal["input_image"]`

873 

874 The type of the image input. Always `input_image`.

875 

876 - `"input_image"`

877 

878 - `detail: Optional[str]`

879 

880 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

881 

882 - `class ResponseInputAudio: …`

883 

884 An audio input to the model.

885 

886 - `input_audio: InputAudio`

887 

888 - `data: str`

889 

890 Base64-encoded audio data.

891 

892 - `format: Literal["mp3", "wav"]`

893 

894 The format of the audio data. Currently supported formats are `mp3` and

895 `wav`.

896 

897 - `"mp3"`

898 

899 - `"wav"`

900 

901 - `type: Literal["input_audio"]`

902 

903 The type of the input item. Always `input_audio`.

904 

905 - `"input_audio"`

906 

907 - `List[GraderInputItem]`

908 

909 - `str`

910 

911 A text input to the model.

912 

913 - `class ResponseInputText: …`

914 

915 A text input to the model.

916 

917 - `class GraderInputItemOutputText: …`

918 

919 A text output from the model.

920 

921 - `text: str`

922 

923 The text output from the model.

924 

925 - `type: Literal["output_text"]`

926 

927 The type of the output text. Always `output_text`.

928 

929 - `"output_text"`

930 

931 - `class GraderInputItemInputImage: …`

932 

933 An image input block used within EvalItem content arrays.

934 

935 - `image_url: str`

936 

937 The URL of the image input.

938 

939 - `type: Literal["input_image"]`

940 

941 The type of the image input. Always `input_image`.

942 

943 - `"input_image"`

944 

945 - `detail: Optional[str]`

946 

947 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

948 

949 - `class ResponseInputAudio: …`

950 

951 An audio input to the model.

952 

953 - `role: Literal["user", "assistant", "system", "developer"]`

954 

955 The role of the message input. One of `user`, `assistant`, `system`, or

956 `developer`.

957 

958 - `"user"`

959 

960 - `"assistant"`

961 

962 - `"system"`

963 

964 - `"developer"`

965 

966 - `type: Optional[Literal["message"]]`

967 

968 The type of the message input. Always `message`.

969 

970 - `"message"`

971 

972 - `model: str`

973 

974 The model to use for the evaluation.

975 

976 - `name: str`

977 

978 The name of the grader.

979 

980 - `type: Literal["score_model"]`

981 

982 The object type, which is always `score_model`.

983 

984 - `"score_model"`

985 

986 - `range: Optional[List[float]]`

987 

988 The range of the score. Defaults to `[0, 1]`.

989 

990 - `sampling_params: Optional[SamplingParams]`

991 

992 The sampling parameters for the model.

993 

994 - `max_completions_tokens: Optional[int]`

995 

996 The maximum number of tokens the grader model may generate in its response.

997 

998 - `reasoning_effort: Optional[ReasoningEffort]`

999 

1000 Constrains effort on reasoning for

1001 [reasoning models](https://platform.openai.com/docs/guides/reasoning).

1002 Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing

1003 reasoning effort can result in faster responses and fewer tokens used

1004 on reasoning in a response.

1005 

1006 - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1.

1007 - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`.

1008 - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort.

1009 - `xhigh` is supported for all models after `gpt-5.1-codex-max`.

1010 

1011 - `"none"`

1012 

1013 - `"minimal"`

1014 

1015 - `"low"`

1016 

1017 - `"medium"`

1018 

1019 - `"high"`

1020 

1021 - `"xhigh"`

1022 

1023 - `seed: Optional[int]`

1024 

1025 A seed value to initialize the randomness, during sampling.

1026 

1027 - `temperature: Optional[float]`

1028 

1029 A higher temperature increases randomness in the outputs.

1030 

1031 - `top_p: Optional[float]`

1032 

1033 An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

1034 

1035 - `class MultiGrader: …`

1036 

1037 A MultiGrader object combines the output of multiple graders to produce a single score.

1038 

1039 - `calculate_output: str`

1040 

1041 A formula to calculate the output based on grader results.

1042 

1043 - `graders: Graders`

1044 

1045 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.

1046 

1047 - `class StringCheckGrader: …`

1048 

1049 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.

1050 

1051 - `class TextSimilarityGrader: …`

1052 

1053 A TextSimilarityGrader object which grades text based on similarity metrics.

1054 

1055 - `class PythonGrader: …`

1056 

1057 A PythonGrader object that runs a python script on the input.

1058 

1059 - `class ScoreModelGrader: …`

1060 

1061 A ScoreModelGrader object that uses a model to assign a score to the input.

1062 

1063 - `class LabelModelGrader: …`

1064 

1065 A LabelModelGrader object which uses a model to assign labels to each item

1066 in the evaluation.

1067 

1068 - `input: List[Input]`

1069 

1070 - `content: InputContent`

1071 

1072 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

1073 

1074 - `str`

1075 

1076 A text input to the model.

1077 

1078 - `class ResponseInputText: …`

1079 

1080 A text input to the model.

1081 

1082 - `class InputContentOutputText: …`

1083 

1084 A text output from the model.

1085 

1086 - `text: str`

1087 

1088 The text output from the model.

1089 

1090 - `type: Literal["output_text"]`

1091 

1092 The type of the output text. Always `output_text`.

1093 

1094 - `"output_text"`

1095 

1096 - `class InputContentInputImage: …`

1097 

1098 An image input block used within EvalItem content arrays.

1099 

1100 - `image_url: str`

1101 

1102 The URL of the image input.

1103 

1104 - `type: Literal["input_image"]`

1105 

1106 The type of the image input. Always `input_image`.

1107 

1108 - `"input_image"`

1109 

1110 - `detail: Optional[str]`

1111 

1112 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

1113 

1114 - `class ResponseInputAudio: …`

1115 

1116 An audio input to the model.

1117 

1118 - `List[GraderInputItem]`

1119 

1120 - `str`

1121 

1122 A text input to the model.

1123 

1124 - `class ResponseInputText: …`

1125 

1126 A text input to the model.

1127 

1128 - `class GraderInputItemOutputText: …`

1129 

1130 A text output from the model.

1131 

1132 - `class GraderInputItemInputImage: …`

1133 

1134 An image input block used within EvalItem content arrays.

1135 

1136 - `class ResponseInputAudio: …`

1137 

1138 An audio input to the model.

1139 

1140 - `role: Literal["user", "assistant", "system", "developer"]`

1141 

1142 The role of the message input. One of `user`, `assistant`, `system`, or

1143 `developer`.

1144 

1145 - `"user"`

1146 

1147 - `"assistant"`

1148 

1149 - `"system"`

1150 

1151 - `"developer"`

1152 

1153 - `type: Optional[Literal["message"]]`

1154 

1155 The type of the message input. Always `message`.

1156 

1157 - `"message"`

1158 

1159 - `labels: List[str]`

1160 

1161 The labels to assign to each item in the evaluation.

1162 

1163 - `model: str`

1164 

1165 The model to use for the evaluation. Must support structured outputs.

1166 

1167 - `name: str`

1168 

1169 The name of the grader.

1170 

1171 - `passing_labels: List[str]`

1172 

1173 The labels that indicate a passing result. Must be a subset of labels.

1174 

1175 - `type: Literal["label_model"]`

1176 

1177 The object type, which is always `label_model`.

1178 

1179 - `"label_model"`

1180 

1181 - `name: str`

1182 

1183 The name of the grader.

1184 

1185 - `type: Literal["multi"]`

1186 

1187 The object type, which is always `multi`.

1188 

1189 - `"multi"`

1190 

1191### Returns

1192 

1193- `class GraderValidateResponse: …`

1194 

1195 - `grader: Optional[Grader]`

1196 

1197 The grader used for the fine-tuning job.

1198 

1199 - `class StringCheckGrader: …`

1200 

1201 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.

1202 

1203 - `input: str`

1204 

1205 The input text. This may include template strings.

1206 

1207 - `name: str`

1208 

1209 The name of the grader.

1210 

1211 - `operation: Literal["eq", "ne", "like", "ilike"]`

1212 

1213 The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.

1214 

1215 - `"eq"`

1216 

1217 - `"ne"`

1218 

1219 - `"like"`

1220 

1221 - `"ilike"`

1222 

1223 - `reference: str`

1224 

1225 The reference text. This may include template strings.

1226 

1227 - `type: Literal["string_check"]`

1228 

1229 The object type, which is always `string_check`.

1230 

1231 - `"string_check"`

1232 

1233 - `class TextSimilarityGrader: …`

1234 

1235 A TextSimilarityGrader object which grades text based on similarity metrics.

1236 

1237 - `evaluation_metric: Literal["cosine", "fuzzy_match", "bleu", 8 more]`

1238 

1239 The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`,

1240 `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`,

1241 or `rouge_l`.

1242 

1243 - `"cosine"`

1244 

1245 - `"fuzzy_match"`

1246 

1247 - `"bleu"`

1248 

1249 - `"gleu"`

1250 

1251 - `"meteor"`

1252 

1253 - `"rouge_1"`

1254 

1255 - `"rouge_2"`

1256 

1257 - `"rouge_3"`

1258 

1259 - `"rouge_4"`

1260 

1261 - `"rouge_5"`

1262 

1263 - `"rouge_l"`

1264 

1265 - `input: str`

1266 

1267 The text being graded.

1268 

1269 - `name: str`

1270 

1271 The name of the grader.

1272 

1273 - `reference: str`

1274 

1275 The text being graded against.

1276 

1277 - `type: Literal["text_similarity"]`

1278 

1279 The type of grader.

1280 

1281 - `"text_similarity"`

1282 

1283 - `class PythonGrader: …`

1284 

1285 A PythonGrader object that runs a python script on the input.

1286 

1287 - `name: str`

1288 

1289 The name of the grader.

1290 

1291 - `source: str`

1292 

1293 The source code of the python script.

1294 

1295 - `type: Literal["python"]`

1296 

1297 The object type, which is always `python`.

1298 

1299 - `"python"`

1300 

1301 - `image_tag: Optional[str]`

1302 

1303 The image tag to use for the python script.

1304 

1305 - `class ScoreModelGrader: …`

1306 

1307 A ScoreModelGrader object that uses a model to assign a score to the input.

1308 

1309 - `input: List[Input]`

1310 

1311 The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.

1312 

1313 - `content: InputContent`

1314 

1315 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

1316 

1317 - `str`

1318 

1319 A text input to the model.

1320 

1321 - `class ResponseInputText: …`

1322 

1323 A text input to the model.

1324 

1325 - `text: str`

1326 

1327 The text input to the model.

1328 

1329 - `type: Literal["input_text"]`

1330 

1331 The type of the input item. Always `input_text`.

1332 

1333 - `"input_text"`

1334 

1335 - `class InputContentOutputText: …`

1336 

1337 A text output from the model.

1338 

1339 - `text: str`

1340 

1341 The text output from the model.

1342 

1343 - `type: Literal["output_text"]`

1344 

1345 The type of the output text. Always `output_text`.

1346 

1347 - `"output_text"`

1348 

1349 - `class InputContentInputImage: …`

1350 

1351 An image input block used within EvalItem content arrays.

1352 

1353 - `image_url: str`

1354 

1355 The URL of the image input.

1356 

1357 - `type: Literal["input_image"]`

1358 

1359 The type of the image input. Always `input_image`.

1360 

1361 - `"input_image"`

1362 

1363 - `detail: Optional[str]`

1364 

1365 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

1366 

1367 - `class ResponseInputAudio: …`

1368 

1369 An audio input to the model.

1370 

1371 - `input_audio: InputAudio`

1372 

1373 - `data: str`

1374 

1375 Base64-encoded audio data.

1376 

1377 - `format: Literal["mp3", "wav"]`

1378 

1379 The format of the audio data. Currently supported formats are `mp3` and

1380 `wav`.

1381 

1382 - `"mp3"`

1383 

1384 - `"wav"`

1385 

1386 - `type: Literal["input_audio"]`

1387 

1388 The type of the input item. Always `input_audio`.

1389 

1390 - `"input_audio"`

1391 

1392 - `List[GraderInputItem]`

1393 

1394 - `str`

1395 

1396 A text input to the model.

1397 

1398 - `class ResponseInputText: …`

1399 

1400 A text input to the model.

1401 

1402 - `class GraderInputItemOutputText: …`

1403 

1404 A text output from the model.

1405 

1406 - `text: str`

1407 

1408 The text output from the model.

1409 

1410 - `type: Literal["output_text"]`

1411 

1412 The type of the output text. Always `output_text`.

1413 

1414 - `"output_text"`

1415 

1416 - `class GraderInputItemInputImage: …`

1417 

1418 An image input block used within EvalItem content arrays.

1419 

1420 - `image_url: str`

1421 

1422 The URL of the image input.

1423 

1424 - `type: Literal["input_image"]`

1425 

1426 The type of the image input. Always `input_image`.

1427 

1428 - `"input_image"`

1429 

1430 - `detail: Optional[str]`

1431 

1432 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

1433 

1434 - `class ResponseInputAudio: …`

1435 

1436 An audio input to the model.

1437 

1438 - `role: Literal["user", "assistant", "system", "developer"]`

1439 

1440 The role of the message input. One of `user`, `assistant`, `system`, or

1441 `developer`.

1442 

1443 - `"user"`

1444 

1445 - `"assistant"`

1446 

1447 - `"system"`

1448 

1449 - `"developer"`

1450 

1451 - `type: Optional[Literal["message"]]`

1452 

1453 The type of the message input. Always `message`.

1454 

1455 - `"message"`

1456 

1457 - `model: str`

1458 

1459 The model to use for the evaluation.

1460 

1461 - `name: str`

1462 

1463 The name of the grader.

1464 

1465 - `type: Literal["score_model"]`

1466 

1467 The object type, which is always `score_model`.

1468 

1469 - `"score_model"`

1470 

1471 - `range: Optional[List[float]]`

1472 

1473 The range of the score. Defaults to `[0, 1]`.

1474 

1475 - `sampling_params: Optional[SamplingParams]`

1476 

1477 The sampling parameters for the model.

1478 

1479 - `max_completions_tokens: Optional[int]`

1480 

1481 The maximum number of tokens the grader model may generate in its response.

1482 

1483 - `reasoning_effort: Optional[ReasoningEffort]`

1484 

1485 Constrains effort on reasoning for

1486 [reasoning models](https://platform.openai.com/docs/guides/reasoning).

1487 Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing

1488 reasoning effort can result in faster responses and fewer tokens used

1489 on reasoning in a response.

1490 

1491 - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1.

1492 - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`.

1493 - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort.

1494 - `xhigh` is supported for all models after `gpt-5.1-codex-max`.

1495 

1496 - `"none"`

1497 

1498 - `"minimal"`

1499 

1500 - `"low"`

1501 

1502 - `"medium"`

1503 

1504 - `"high"`

1505 

1506 - `"xhigh"`

1507 

1508 - `seed: Optional[int]`

1509 

1510 A seed value to initialize the randomness, during sampling.

1511 

1512 - `temperature: Optional[float]`

1513 

1514 A higher temperature increases randomness in the outputs.

1515 

1516 - `top_p: Optional[float]`

1517 

1518 An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

1519 

1520 - `class MultiGrader: …`

1521 

1522 A MultiGrader object combines the output of multiple graders to produce a single score.

1523 

1524 - `calculate_output: str`

1525 

1526 A formula to calculate the output based on grader results.

1527 

1528 - `graders: Graders`

1529 

1530 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.

1531 

1532 - `class StringCheckGrader: …`

1533 

1534 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.

1535 

1536 - `class TextSimilarityGrader: …`

1537 

1538 A TextSimilarityGrader object which grades text based on similarity metrics.

1539 

1540 - `class PythonGrader: …`

1541 

1542 A PythonGrader object that runs a python script on the input.

1543 

1544 - `class ScoreModelGrader: …`

1545 

1546 A ScoreModelGrader object that uses a model to assign a score to the input.

1547 

1548 - `class LabelModelGrader: …`

1549 

1550 A LabelModelGrader object which uses a model to assign labels to each item

1551 in the evaluation.

1552 

1553 - `input: List[Input]`

1554 

1555 - `content: InputContent`

1556 

1557 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

1558 

1559 - `str`

1560 

1561 A text input to the model.

1562 

1563 - `class ResponseInputText: …`

1564 

1565 A text input to the model.

1566 

1567 - `class InputContentOutputText: …`

1568 

1569 A text output from the model.

1570 

1571 - `text: str`

1572 

1573 The text output from the model.

1574 

1575 - `type: Literal["output_text"]`

1576 

1577 The type of the output text. Always `output_text`.

1578 

1579 - `"output_text"`

1580 

1581 - `class InputContentInputImage: …`

1582 

1583 An image input block used within EvalItem content arrays.

1584 

1585 - `image_url: str`

1586 

1587 The URL of the image input.

1588 

1589 - `type: Literal["input_image"]`

1590 

1591 The type of the image input. Always `input_image`.

1592 

1593 - `"input_image"`

1594 

1595 - `detail: Optional[str]`

1596 

1597 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

1598 

1599 - `class ResponseInputAudio: …`

1600 

1601 An audio input to the model.

1602 

1603 - `List[GraderInputItem]`

1604 

1605 - `str`

1606 

1607 A text input to the model.

1608 

1609 - `class ResponseInputText: …`

1610 

1611 A text input to the model.

1612 

1613 - `class GraderInputItemOutputText: …`

1614 

1615 A text output from the model.

1616 

1617 - `class GraderInputItemInputImage: …`

1618 

1619 An image input block used within EvalItem content arrays.

1620 

1621 - `class ResponseInputAudio: …`

1622 

1623 An audio input to the model.

1624 

1625 - `role: Literal["user", "assistant", "system", "developer"]`

1626 

1627 The role of the message input. One of `user`, `assistant`, `system`, or

1628 `developer`.

1629 

1630 - `"user"`

1631 

1632 - `"assistant"`

1633 

1634 - `"system"`

1635 

1636 - `"developer"`

1637 

1638 - `type: Optional[Literal["message"]]`

1639 

1640 The type of the message input. Always `message`.

1641 

1642 - `"message"`

1643 

1644 - `labels: List[str]`

1645 

1646 The labels to assign to each item in the evaluation.

1647 

1648 - `model: str`

1649 

1650 The model to use for the evaluation. Must support structured outputs.

1651 

1652 - `name: str`

1653 

1654 The name of the grader.

1655 

1656 - `passing_labels: List[str]`

1657 

1658 The labels that indicate a passing result. Must be a subset of labels.

1659 

1660 - `type: Literal["label_model"]`

1661 

1662 The object type, which is always `label_model`.

1663 

1664 - `"label_model"`

1665 

1666 - `name: str`

1667 

1668 The name of the grader.

1669 

1670 - `type: Literal["multi"]`

1671 

1672 The object type, which is always `multi`.

1673 

1674 - `"multi"`

1675 

1676### Example

1677 

1678```python

1679import os

1680from openai import OpenAI

1681 

1682client = OpenAI(

1683 api_key=os.environ.get("OPENAI_API_KEY"), # This is the default and can be omitted

1684)

1685response = client.fine_tuning.alpha.graders.validate(

1686 grader={

1687 "input": "input",

1688 "name": "name",

1689 "operation": "eq",

1690 "reference": "reference",

1691 "type": "string_check",

1692 },

1693)

1694print(response.grader)

1695```

1696 

1697#### Response

1698 

1699```json

1700{

1701 "grader": {

1702 "input": "input",

1703 "name": "name",

1704 "operation": "eq",

1705 "reference": "reference",

1706 "type": "string_check"

1707 }

1708}

1709```

1710 

1711## Domain Types

1712 

1713### Grader Run Response

1714 

1715- `class GraderRunResponse: …`

1716 

1717 - `metadata: Metadata`

1718 

1719 - `errors: MetadataErrors`

1720 

1721 - `formula_parse_error: bool`

1722 

1723 - `invalid_variable_error: bool`

1724 

1725 - `model_grader_parse_error: bool`

1726 

1727 - `model_grader_refusal_error: bool`

1728 

1729 - `model_grader_server_error: bool`

1730 

1731 - `model_grader_server_error_details: Optional[str]`

1732 

1733 - `other_error: bool`

1734 

1735 - `python_grader_runtime_error: bool`

1736 

1737 - `python_grader_runtime_error_details: Optional[str]`

1738 

1739 - `python_grader_server_error: bool`

1740 

1741 - `python_grader_server_error_type: Optional[str]`

1742 

1743 - `sample_parse_error: bool`

1744 

1745 - `truncated_observation_error: bool`

1746 

1747 - `unresponsive_reward_error: bool`

1748 

1749 - `execution_time: float`

1750 

1751 - `name: str`

1752 

1753 - `sampled_model_name: Optional[str]`

1754 

1755 - `scores: Dict[str, object]`

1756 

1757 - `token_usage: Optional[int]`

1758 

1759 - `type: str`

1760 

1761 - `model_grader_token_usage_per_model: Dict[str, object]`

1762 

1763 - `reward: float`

1764 

1765 - `sub_rewards: Dict[str, object]`

1766 

1767### Grader Validate Response

1768 

1769- `class GraderValidateResponse: …`

1770 

1771 - `grader: Optional[Grader]`

1772 

1773 The grader used for the fine-tuning job.

1774 

1775 - `class StringCheckGrader: …`

1776 

1777 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.

1778 

1779 - `input: str`

1780 

1781 The input text. This may include template strings.

1782 

1783 - `name: str`

1784 

1785 The name of the grader.

1786 

1787 - `operation: Literal["eq", "ne", "like", "ilike"]`

1788 

1789 The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.

1790 

1791 - `"eq"`

1792 

1793 - `"ne"`

1794 

1795 - `"like"`

1796 

1797 - `"ilike"`

1798 

1799 - `reference: str`

1800 

1801 The reference text. This may include template strings.

1802 

1803 - `type: Literal["string_check"]`

1804 

1805 The object type, which is always `string_check`.

1806 

1807 - `"string_check"`

1808 

1809 - `class TextSimilarityGrader: …`

1810 

1811 A TextSimilarityGrader object which grades text based on similarity metrics.

1812 

1813 - `evaluation_metric: Literal["cosine", "fuzzy_match", "bleu", 8 more]`

1814 

1815 The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`,

1816 `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`,

1817 or `rouge_l`.

1818 

1819 - `"cosine"`

1820 

1821 - `"fuzzy_match"`

1822 

1823 - `"bleu"`

1824 

1825 - `"gleu"`

1826 

1827 - `"meteor"`

1828 

1829 - `"rouge_1"`

1830 

1831 - `"rouge_2"`

1832 

1833 - `"rouge_3"`

1834 

1835 - `"rouge_4"`

1836 

1837 - `"rouge_5"`

1838 

1839 - `"rouge_l"`

1840 

1841 - `input: str`

1842 

1843 The text being graded.

1844 

1845 - `name: str`

1846 

1847 The name of the grader.

1848 

1849 - `reference: str`

1850 

1851 The text being graded against.

1852 

1853 - `type: Literal["text_similarity"]`

1854 

1855 The type of grader.

1856 

1857 - `"text_similarity"`

1858 

1859 - `class PythonGrader: …`

1860 

1861 A PythonGrader object that runs a python script on the input.

1862 

1863 - `name: str`

1864 

1865 The name of the grader.

1866 

1867 - `source: str`

1868 

1869 The source code of the python script.

1870 

1871 - `type: Literal["python"]`

1872 

1873 The object type, which is always `python`.

1874 

1875 - `"python"`

1876 

1877 - `image_tag: Optional[str]`

1878 

1879 The image tag to use for the python script.

1880 

1881 - `class ScoreModelGrader: …`

1882 

1883 A ScoreModelGrader object that uses a model to assign a score to the input.

1884 

1885 - `input: List[Input]`

1886 

1887 The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.

1888 

1889 - `content: InputContent`

1890 

1891 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

1892 

1893 - `str`

1894 

1895 A text input to the model.

1896 

1897 - `class ResponseInputText: …`

1898 

1899 A text input to the model.

1900 

1901 - `text: str`

1902 

1903 The text input to the model.

1904 

1905 - `type: Literal["input_text"]`

1906 

1907 The type of the input item. Always `input_text`.

1908 

1909 - `"input_text"`

1910 

1911 - `class InputContentOutputText: …`

1912 

1913 A text output from the model.

1914 

1915 - `text: str`

1916 

1917 The text output from the model.

1918 

1919 - `type: Literal["output_text"]`

1920 

1921 The type of the output text. Always `output_text`.

1922 

1923 - `"output_text"`

1924 

1925 - `class InputContentInputImage: …`

1926 

1927 An image input block used within EvalItem content arrays.

1928 

1929 - `image_url: str`

1930 

1931 The URL of the image input.

1932 

1933 - `type: Literal["input_image"]`

1934 

1935 The type of the image input. Always `input_image`.

1936 

1937 - `"input_image"`

1938 

1939 - `detail: Optional[str]`

1940 

1941 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

1942 

1943 - `class ResponseInputAudio: …`

1944 

1945 An audio input to the model.

1946 

1947 - `input_audio: InputAudio`

1948 

1949 - `data: str`

1950 

1951 Base64-encoded audio data.

1952 

1953 - `format: Literal["mp3", "wav"]`

1954 

1955 The format of the audio data. Currently supported formats are `mp3` and

1956 `wav`.

1957 

1958 - `"mp3"`

1959 

1960 - `"wav"`

1961 

1962 - `type: Literal["input_audio"]`

1963 

1964 The type of the input item. Always `input_audio`.

1965 

1966 - `"input_audio"`

1967 

1968 - `List[GraderInputItem]`

1969 

1970 - `str`

1971 

1972 A text input to the model.

1973 

1974 - `class ResponseInputText: …`

1975 

1976 A text input to the model.

1977 

1978 - `class GraderInputItemOutputText: …`

1979 

1980 A text output from the model.

1981 

1982 - `text: str`

1983 

1984 The text output from the model.

1985 

1986 - `type: Literal["output_text"]`

1987 

1988 The type of the output text. Always `output_text`.

1989 

1990 - `"output_text"`

1991 

1992 - `class GraderInputItemInputImage: …`

1993 

1994 An image input block used within EvalItem content arrays.

1995 

1996 - `image_url: str`