java/resources/fine_tuning/subresources/alpha/index.md +0 −1607 deleted
File Deleted View Diff
1# Alpha
2
3# Graders
4
5## Run grader
6
7`GraderRunResponse fineTuning().alpha().graders().run(GraderRunParamsparams, RequestOptionsrequestOptions = RequestOptions.none())`
8
9**post** `/fine_tuning/alpha/graders/run`
10
11Run a grader.
12
13### Parameters
14
15- `GraderRunParams params`
16
17 - `Grader grader`
18
19 The grader used for the fine-tuning job.
20
21 - `class StringCheckGrader:`
22
23 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
24
25 - `String input`
26
27 The input text. This may include template strings.
28
29 - `String name`
30
31 The name of the grader.
32
33 - `Operation operation`
34
35 The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
36
37 - `EQ("eq")`
38
39 - `NE("ne")`
40
41 - `LIKE("like")`
42
43 - `ILIKE("ilike")`
44
45 - `String reference`
46
47 The reference text. This may include template strings.
48
49 - `JsonValue; type "string_check"constant`
50
51 The object type, which is always `string_check`.
52
53 - `STRING_CHECK("string_check")`
54
55 - `class TextSimilarityGrader:`
56
57 A TextSimilarityGrader object which grades text based on similarity metrics.
58
59 - `EvaluationMetric evaluationMetric`
60
61 The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`,
62 `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`,
63 or `rouge_l`.
64
65 - `COSINE("cosine")`
66
67 - `FUZZY_MATCH("fuzzy_match")`
68
69 - `BLEU("bleu")`
70
71 - `GLEU("gleu")`
72
73 - `METEOR("meteor")`
74
75 - `ROUGE_1("rouge_1")`
76
77 - `ROUGE_2("rouge_2")`
78
79 - `ROUGE_3("rouge_3")`
80
81 - `ROUGE_4("rouge_4")`
82
83 - `ROUGE_5("rouge_5")`
84
85 - `ROUGE_L("rouge_l")`
86
87 - `String input`
88
89 The text being graded.
90
91 - `String name`
92
93 The name of the grader.
94
95 - `String reference`
96
97 The text being graded against.
98
99 - `JsonValue; type "text_similarity"constant`
100
101 The type of grader.
102
103 - `TEXT_SIMILARITY("text_similarity")`
104
105 - `class PythonGrader:`
106
107 A PythonGrader object that runs a python script on the input.
108
109 - `String name`
110
111 The name of the grader.
112
113 - `String source`
114
115 The source code of the python script.
116
117 - `JsonValue; type "python"constant`
118
119 The object type, which is always `python`.
120
121 - `PYTHON("python")`
122
123 - `Optional<String> imageTag`
124
125 The image tag to use for the python script.
126
127 - `class ScoreModelGrader:`
128
129 A ScoreModelGrader object that uses a model to assign a score to the input.
130
131 - `List<Input> input`
132
133 The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.
134
135 - `Content content`
136
137 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.
138
139 - `String`
140
141 - `class ResponseInputText:`
142
143 A text input to the model.
144
145 - `String text`
146
147 The text input to the model.
148
149 - `JsonValue; type "input_text"constant`
150
151 The type of the input item. Always `input_text`.
152
153 - `INPUT_TEXT("input_text")`
154
155 - `class OutputText:`
156
157 A text output from the model.
158
159 - `String text`
160
161 The text output from the model.
162
163 - `JsonValue; type "output_text"constant`
164
165 The type of the output text. Always `output_text`.
166
167 - `OUTPUT_TEXT("output_text")`
168
169 - `class InputImage:`
170
171 An image input block used within EvalItem content arrays.
172
173 - `String imageUrl`
174
175 The URL of the image input.
176
177 - `JsonValue; type "input_image"constant`
178
179 The type of the image input. Always `input_image`.
180
181 - `INPUT_IMAGE("input_image")`
182
183 - `Optional<String> detail`
184
185 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
186
187 - `class ResponseInputAudio:`
188
189 An audio input to the model.
190
191 - `InputAudio inputAudio`
192
193 - `String data`
194
195 Base64-encoded audio data.
196
197 - `Format format`
198
199 The format of the audio data. Currently supported formats are `mp3` and
200 `wav`.
201
202 - `MP3("mp3")`
203
204 - `WAV("wav")`
205
206 - `JsonValue; type "input_audio"constant`
207
208 The type of the input item. Always `input_audio`.
209
210 - `INPUT_AUDIO("input_audio")`
211
212 - `List<EvalContentItem>`
213
214 - `String`
215
216 - `class ResponseInputText:`
217
218 A text input to the model.
219
220 - `OutputText`
221
222 - `String text`
223
224 The text output from the model.
225
226 - `JsonValue; type "output_text"constant`
227
228 The type of the output text. Always `output_text`.
229
230 - `OUTPUT_TEXT("output_text")`
231
232 - `InputImage`
233
234 - `String imageUrl`
235
236 The URL of the image input.
237
238 - `JsonValue; type "input_image"constant`
239
240 The type of the image input. Always `input_image`.
241
242 - `INPUT_IMAGE("input_image")`
243
244 - `Optional<String> detail`
245
246 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
247
248 - `class ResponseInputAudio:`
249
250 An audio input to the model.
251
252 - `Role role`
253
254 The role of the message input. One of `user`, `assistant`, `system`, or
255 `developer`.
256
257 - `USER("user")`
258
259 - `ASSISTANT("assistant")`
260
261 - `SYSTEM("system")`
262
263 - `DEVELOPER("developer")`
264
265 - `Optional<Type> type`
266
267 The type of the message input. Always `message`.
268
269 - `MESSAGE("message")`
270
271 - `String model`
272
273 The model to use for the evaluation.
274
275 - `String name`
276
277 The name of the grader.
278
279 - `JsonValue; type "score_model"constant`
280
281 The object type, which is always `score_model`.
282
283 - `SCORE_MODEL("score_model")`
284
285 - `Optional<List<Double>> range`
286
287 The range of the score. Defaults to `[0, 1]`.
288
289 - `Optional<SamplingParams> samplingParams`
290
291 The sampling parameters for the model.
292
293 - `Optional<Long> maxCompletionsTokens`
294
295 The maximum number of tokens the grader model may generate in its response.
296
297 - `Optional<ReasoningEffort> reasoningEffort`
298
299 Constrains effort on reasoning for
300 [reasoning models](https://platform.openai.com/docs/guides/reasoning).
301 Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing
302 reasoning effort can result in faster responses and fewer tokens used
303 on reasoning in a response.
304
305 - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1.
306 - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`.
307 - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort.
308 - `xhigh` is supported for all models after `gpt-5.1-codex-max`.
309
310 - `NONE("none")`
311
312 - `MINIMAL("minimal")`
313
314 - `LOW("low")`
315
316 - `MEDIUM("medium")`
317
318 - `HIGH("high")`
319
320 - `XHIGH("xhigh")`
321
322 - `Optional<Long> seed`
323
324 A seed value to initialize the randomness, during sampling.
325
326 - `Optional<Double> temperature`
327
328 A higher temperature increases randomness in the outputs.
329
330 - `Optional<Double> topP`
331
332 An alternative to temperature for nucleus sampling; 1.0 includes all tokens.
333
334 - `class MultiGrader:`
335
336 A MultiGrader object combines the output of multiple graders to produce a single score.
337
338 - `String calculateOutput`
339
340 A formula to calculate the output based on grader results.
341
342 - `Graders graders`
343
344 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
345
346 - `class StringCheckGrader:`
347
348 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
349
350 - `class TextSimilarityGrader:`
351
352 A TextSimilarityGrader object which grades text based on similarity metrics.
353
354 - `class PythonGrader:`
355
356 A PythonGrader object that runs a python script on the input.
357
358 - `class ScoreModelGrader:`
359
360 A ScoreModelGrader object that uses a model to assign a score to the input.
361
362 - `class LabelModelGrader:`
363
364 A LabelModelGrader object which uses a model to assign labels to each item
365 in the evaluation.
366
367 - `List<Input> input`
368
369 - `Content content`
370
371 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.
372
373 - `String`
374
375 - `class ResponseInputText:`
376
377 A text input to the model.
378
379 - `class OutputText:`
380
381 A text output from the model.
382
383 - `String text`
384
385 The text output from the model.
386
387 - `JsonValue; type "output_text"constant`
388
389 The type of the output text. Always `output_text`.
390
391 - `OUTPUT_TEXT("output_text")`
392
393 - `class InputImage:`
394
395 An image input block used within EvalItem content arrays.
396
397 - `String imageUrl`
398
399 The URL of the image input.
400
401 - `JsonValue; type "input_image"constant`
402
403 The type of the image input. Always `input_image`.
404
405 - `INPUT_IMAGE("input_image")`
406
407 - `Optional<String> detail`
408
409 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
410
411 - `class ResponseInputAudio:`
412
413 An audio input to the model.
414
415 - `List<EvalContentItem>`
416
417 - `String`
418
419 - `class ResponseInputText:`
420
421 A text input to the model.
422
423 - `OutputText`
424
425 - `InputImage`
426
427 - `class ResponseInputAudio:`
428
429 An audio input to the model.
430
431 - `Role role`
432
433 The role of the message input. One of `user`, `assistant`, `system`, or
434 `developer`.
435
436 - `USER("user")`
437
438 - `ASSISTANT("assistant")`
439
440 - `SYSTEM("system")`
441
442 - `DEVELOPER("developer")`
443
444 - `Optional<Type> type`
445
446 The type of the message input. Always `message`.
447
448 - `MESSAGE("message")`
449
450 - `List<String> labels`
451
452 The labels to assign to each item in the evaluation.
453
454 - `String model`
455
456 The model to use for the evaluation. Must support structured outputs.
457
458 - `String name`
459
460 The name of the grader.
461
462 - `List<String> passingLabels`
463
464 The labels that indicate a passing result. Must be a subset of labels.
465
466 - `JsonValue; type "label_model"constant`
467
468 The object type, which is always `label_model`.
469
470 - `LABEL_MODEL("label_model")`
471
472 - `String name`
473
474 The name of the grader.
475
476 - `JsonValue; type "multi"constant`
477
478 The object type, which is always `multi`.
479
480 - `MULTI("multi")`
481
482 - `String modelSample`
483
484 The model sample to be evaluated. This value will be used to populate
485 the `sample` namespace. See [the guide](https://platform.openai.com/docs/guides/graders) for more details.
486 The `output_json` variable will be populated if the model sample is a
487 valid JSON string.
488
489 - `Optional<JsonValue> item`
490
491 The dataset item provided to the grader. This will be used to populate
492 the `item` namespace. See [the guide](https://platform.openai.com/docs/guides/graders) for more details.
493
494### Returns
495
496- `class GraderRunResponse:`
497
498 - `Metadata metadata`
499
500 - `Errors errors`
501
502 - `boolean formulaParseError`
503
504 - `boolean invalidVariableError`
505
506 - `boolean modelGraderParseError`
507
508 - `boolean modelGraderRefusalError`
509
510 - `boolean modelGraderServerError`
511
512 - `Optional<String> modelGraderServerErrorDetails`
513
514 - `boolean otherError`
515
516 - `boolean pythonGraderRuntimeError`
517
518 - `Optional<String> pythonGraderRuntimeErrorDetails`
519
520 - `boolean pythonGraderServerError`
521
522 - `Optional<String> pythonGraderServerErrorType`
523
524 - `boolean sampleParseError`
525
526 - `boolean truncatedObservationError`
527
528 - `boolean unresponsiveRewardError`
529
530 - `double executionTime`
531
532 - `String name`
533
534 - `Optional<String> sampledModelName`
535
536 - `Scores scores`
537
538 - `Optional<Long> tokenUsage`
539
540 - `String type`
541
542 - `ModelGraderTokenUsagePerModel modelGraderTokenUsagePerModel`
543
544 - `double reward`
545
546 - `SubRewards subRewards`
547
548### Example
549
550```java
551package com.openai.example;
552
553import com.openai.client.OpenAIClient;
554import com.openai.client.okhttp.OpenAIOkHttpClient;
555import com.openai.models.finetuning.alpha.graders.GraderRunParams;
556import com.openai.models.finetuning.alpha.graders.GraderRunResponse;
557import com.openai.models.graders.gradermodels.StringCheckGrader;
558
559public final class Main {
560 private Main() {}
561
562 public static void main(String[] args) {
563 OpenAIClient client = OpenAIOkHttpClient.fromEnv();
564
565 GraderRunParams params = GraderRunParams.builder()
566 .grader(StringCheckGrader.builder()
567 .input("input")
568 .name("name")
569 .operation(StringCheckGrader.Operation.EQ)
570 .reference("reference")
571 .build())
572 .modelSample("model_sample")
573 .build();
574 GraderRunResponse response = client.fineTuning().alpha().graders().run(params);
575 }
576}
577```
578
579#### Response
580
581```json
582{
583 "metadata": {
584 "errors": {
585 "formula_parse_error": true,
586 "invalid_variable_error": true,
587 "model_grader_parse_error": true,
588 "model_grader_refusal_error": true,
589 "model_grader_server_error": true,
590 "model_grader_server_error_details": "model_grader_server_error_details",
591 "other_error": true,
592 "python_grader_runtime_error": true,
593 "python_grader_runtime_error_details": "python_grader_runtime_error_details",
594 "python_grader_server_error": true,
595 "python_grader_server_error_type": "python_grader_server_error_type",
596 "sample_parse_error": true,
597 "truncated_observation_error": true,
598 "unresponsive_reward_error": true
599 },
600 "execution_time": 0,
601 "name": "name",
602 "sampled_model_name": "sampled_model_name",
603 "scores": {
604 "foo": "bar"
605 },
606 "token_usage": 0,
607 "type": "type"
608 },
609 "model_grader_token_usage_per_model": {
610 "foo": "bar"
611 },
612 "reward": 0,
613 "sub_rewards": {
614 "foo": "bar"
615 }
616}
617```
618
619## Validate grader
620
621`GraderValidateResponse fineTuning().alpha().graders().validate(GraderValidateParamsparams, RequestOptionsrequestOptions = RequestOptions.none())`
622
623**post** `/fine_tuning/alpha/graders/validate`
624
625Validate a grader.
626
627### Parameters
628
629- `GraderValidateParams params`
630
631 - `Grader grader`
632
633 The grader used for the fine-tuning job.
634
635 - `class StringCheckGrader:`
636
637 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
638
639 - `String input`
640
641 The input text. This may include template strings.
642
643 - `String name`
644
645 The name of the grader.
646
647 - `Operation operation`
648
649 The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
650
651 - `EQ("eq")`
652
653 - `NE("ne")`
654
655 - `LIKE("like")`
656
657 - `ILIKE("ilike")`
658
659 - `String reference`
660
661 The reference text. This may include template strings.
662
663 - `JsonValue; type "string_check"constant`
664
665 The object type, which is always `string_check`.
666
667 - `STRING_CHECK("string_check")`
668
669 - `class TextSimilarityGrader:`
670
671 A TextSimilarityGrader object which grades text based on similarity metrics.
672
673 - `EvaluationMetric evaluationMetric`
674
675 The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`,
676 `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`,
677 or `rouge_l`.
678
679 - `COSINE("cosine")`
680
681 - `FUZZY_MATCH("fuzzy_match")`
682
683 - `BLEU("bleu")`
684
685 - `GLEU("gleu")`
686
687 - `METEOR("meteor")`
688
689 - `ROUGE_1("rouge_1")`
690
691 - `ROUGE_2("rouge_2")`
692
693 - `ROUGE_3("rouge_3")`
694
695 - `ROUGE_4("rouge_4")`
696
697 - `ROUGE_5("rouge_5")`
698
699 - `ROUGE_L("rouge_l")`
700
701 - `String input`
702
703 The text being graded.
704
705 - `String name`
706
707 The name of the grader.
708
709 - `String reference`
710
711 The text being graded against.
712
713 - `JsonValue; type "text_similarity"constant`
714
715 The type of grader.
716
717 - `TEXT_SIMILARITY("text_similarity")`
718
719 - `class PythonGrader:`
720
721 A PythonGrader object that runs a python script on the input.
722
723 - `String name`
724
725 The name of the grader.
726
727 - `String source`
728
729 The source code of the python script.
730
731 - `JsonValue; type "python"constant`
732
733 The object type, which is always `python`.
734
735 - `PYTHON("python")`
736
737 - `Optional<String> imageTag`
738
739 The image tag to use for the python script.
740
741 - `class ScoreModelGrader:`
742
743 A ScoreModelGrader object that uses a model to assign a score to the input.
744
745 - `List<Input> input`
746
747 The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.
748
749 - `Content content`
750
751 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.
752
753 - `String`
754
755 - `class ResponseInputText:`
756
757 A text input to the model.
758
759 - `String text`
760
761 The text input to the model.
762
763 - `JsonValue; type "input_text"constant`
764
765 The type of the input item. Always `input_text`.
766
767 - `INPUT_TEXT("input_text")`
768
769 - `class OutputText:`
770
771 A text output from the model.
772
773 - `String text`
774
775 The text output from the model.
776
777 - `JsonValue; type "output_text"constant`
778
779 The type of the output text. Always `output_text`.
780
781 - `OUTPUT_TEXT("output_text")`
782
783 - `class InputImage:`
784
785 An image input block used within EvalItem content arrays.
786
787 - `String imageUrl`
788
789 The URL of the image input.
790
791 - `JsonValue; type "input_image"constant`
792
793 The type of the image input. Always `input_image`.
794
795 - `INPUT_IMAGE("input_image")`
796
797 - `Optional<String> detail`
798
799 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
800
801 - `class ResponseInputAudio:`
802
803 An audio input to the model.
804
805 - `InputAudio inputAudio`
806
807 - `String data`
808
809 Base64-encoded audio data.
810
811 - `Format format`
812
813 The format of the audio data. Currently supported formats are `mp3` and
814 `wav`.
815
816 - `MP3("mp3")`
817
818 - `WAV("wav")`
819
820 - `JsonValue; type "input_audio"constant`
821
822 The type of the input item. Always `input_audio`.
823
824 - `INPUT_AUDIO("input_audio")`
825
826 - `List<EvalContentItem>`
827
828 - `String`
829
830 - `class ResponseInputText:`
831
832 A text input to the model.
833
834 - `OutputText`
835
836 - `String text`
837
838 The text output from the model.
839
840 - `JsonValue; type "output_text"constant`
841
842 The type of the output text. Always `output_text`.
843
844 - `OUTPUT_TEXT("output_text")`
845
846 - `InputImage`
847
848 - `String imageUrl`
849
850 The URL of the image input.
851
852 - `JsonValue; type "input_image"constant`
853
854 The type of the image input. Always `input_image`.
855
856 - `INPUT_IMAGE("input_image")`
857
858 - `Optional<String> detail`
859
860 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
861
862 - `class ResponseInputAudio:`
863
864 An audio input to the model.
865
866 - `Role role`
867
868 The role of the message input. One of `user`, `assistant`, `system`, or
869 `developer`.
870
871 - `USER("user")`
872
873 - `ASSISTANT("assistant")`
874
875 - `SYSTEM("system")`
876
877 - `DEVELOPER("developer")`
878
879 - `Optional<Type> type`
880
881 The type of the message input. Always `message`.
882
883 - `MESSAGE("message")`
884
885 - `String model`
886
887 The model to use for the evaluation.
888
889 - `String name`
890
891 The name of the grader.
892
893 - `JsonValue; type "score_model"constant`
894
895 The object type, which is always `score_model`.
896
897 - `SCORE_MODEL("score_model")`
898
899 - `Optional<List<Double>> range`
900
901 The range of the score. Defaults to `[0, 1]`.
902
903 - `Optional<SamplingParams> samplingParams`
904
905 The sampling parameters for the model.
906
907 - `Optional<Long> maxCompletionsTokens`
908
909 The maximum number of tokens the grader model may generate in its response.
910
911 - `Optional<ReasoningEffort> reasoningEffort`
912
913 Constrains effort on reasoning for
914 [reasoning models](https://platform.openai.com/docs/guides/reasoning).
915 Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing
916 reasoning effort can result in faster responses and fewer tokens used
917 on reasoning in a response.
918
919 - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1.
920 - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`.
921 - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort.
922 - `xhigh` is supported for all models after `gpt-5.1-codex-max`.
923
924 - `NONE("none")`
925
926 - `MINIMAL("minimal")`
927
928 - `LOW("low")`
929
930 - `MEDIUM("medium")`
931
932 - `HIGH("high")`
933
934 - `XHIGH("xhigh")`
935
936 - `Optional<Long> seed`
937
938 A seed value to initialize the randomness, during sampling.
939
940 - `Optional<Double> temperature`
941
942 A higher temperature increases randomness in the outputs.
943
944 - `Optional<Double> topP`
945
946 An alternative to temperature for nucleus sampling; 1.0 includes all tokens.
947
948 - `class MultiGrader:`
949
950 A MultiGrader object combines the output of multiple graders to produce a single score.
951
952 - `String calculateOutput`
953
954 A formula to calculate the output based on grader results.
955
956 - `Graders graders`
957
958 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
959
960 - `class StringCheckGrader:`
961
962 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
963
964 - `class TextSimilarityGrader:`
965
966 A TextSimilarityGrader object which grades text based on similarity metrics.
967
968 - `class PythonGrader:`
969
970 A PythonGrader object that runs a python script on the input.
971
972 - `class ScoreModelGrader:`
973
974 A ScoreModelGrader object that uses a model to assign a score to the input.
975
976 - `class LabelModelGrader:`
977
978 A LabelModelGrader object which uses a model to assign labels to each item
979 in the evaluation.
980
981 - `List<Input> input`
982
983 - `Content content`
984
985 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.
986
987 - `String`
988
989 - `class ResponseInputText:`
990
991 A text input to the model.
992
993 - `class OutputText:`
994
995 A text output from the model.
996
997 - `String text`
998
999 The text output from the model.
1000
1001 - `JsonValue; type "output_text"constant`
1002
1003 The type of the output text. Always `output_text`.
1004
1005 - `OUTPUT_TEXT("output_text")`
1006
1007 - `class InputImage:`
1008
1009 An image input block used within EvalItem content arrays.
1010
1011 - `String imageUrl`
1012
1013 The URL of the image input.
1014
1015 - `JsonValue; type "input_image"constant`
1016
1017 The type of the image input. Always `input_image`.
1018
1019 - `INPUT_IMAGE("input_image")`
1020
1021 - `Optional<String> detail`
1022
1023 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
1024
1025 - `class ResponseInputAudio:`
1026
1027 An audio input to the model.
1028
1029 - `List<EvalContentItem>`
1030
1031 - `String`
1032
1033 - `class ResponseInputText:`
1034
1035 A text input to the model.
1036
1037 - `OutputText`
1038
1039 - `InputImage`
1040
1041 - `class ResponseInputAudio:`
1042
1043 An audio input to the model.
1044
1045 - `Role role`
1046
1047 The role of the message input. One of `user`, `assistant`, `system`, or
1048 `developer`.
1049
1050 - `USER("user")`
1051
1052 - `ASSISTANT("assistant")`
1053
1054 - `SYSTEM("system")`
1055
1056 - `DEVELOPER("developer")`
1057
1058 - `Optional<Type> type`
1059
1060 The type of the message input. Always `message`.
1061
1062 - `MESSAGE("message")`
1063
1064 - `List<String> labels`
1065
1066 The labels to assign to each item in the evaluation.
1067
1068 - `String model`
1069
1070 The model to use for the evaluation. Must support structured outputs.
1071
1072 - `String name`
1073
1074 The name of the grader.
1075
1076 - `List<String> passingLabels`
1077
1078 The labels that indicate a passing result. Must be a subset of labels.
1079
1080 - `JsonValue; type "label_model"constant`
1081
1082 The object type, which is always `label_model`.
1083
1084 - `LABEL_MODEL("label_model")`
1085
1086 - `String name`
1087
1088 The name of the grader.
1089
1090 - `JsonValue; type "multi"constant`
1091
1092 The object type, which is always `multi`.
1093
1094 - `MULTI("multi")`
1095
1096### Returns
1097
1098- `class GraderValidateResponse:`
1099
1100 - `Optional<Grader> grader`
1101
1102 The grader used for the fine-tuning job.
1103
1104 - `class StringCheckGrader:`
1105
1106 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
1107
1108 - `String input`
1109
1110 The input text. This may include template strings.
1111
1112 - `String name`
1113
1114 The name of the grader.
1115
1116 - `Operation operation`
1117
1118 The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
1119
1120 - `EQ("eq")`
1121
1122 - `NE("ne")`
1123
1124 - `LIKE("like")`
1125
1126 - `ILIKE("ilike")`
1127
1128 - `String reference`
1129
1130 The reference text. This may include template strings.
1131
1132 - `JsonValue; type "string_check"constant`
1133
1134 The object type, which is always `string_check`.
1135
1136 - `STRING_CHECK("string_check")`
1137
1138 - `class TextSimilarityGrader:`
1139
1140 A TextSimilarityGrader object which grades text based on similarity metrics.
1141
1142 - `EvaluationMetric evaluationMetric`
1143
1144 The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`,
1145 `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`,
1146 or `rouge_l`.
1147
1148 - `COSINE("cosine")`
1149
1150 - `FUZZY_MATCH("fuzzy_match")`
1151
1152 - `BLEU("bleu")`
1153
1154 - `GLEU("gleu")`
1155
1156 - `METEOR("meteor")`
1157
1158 - `ROUGE_1("rouge_1")`
1159
1160 - `ROUGE_2("rouge_2")`
1161
1162 - `ROUGE_3("rouge_3")`
1163
1164 - `ROUGE_4("rouge_4")`
1165
1166 - `ROUGE_5("rouge_5")`
1167
1168 - `ROUGE_L("rouge_l")`
1169
1170 - `String input`
1171
1172 The text being graded.
1173
1174 - `String name`
1175
1176 The name of the grader.
1177
1178 - `String reference`
1179
1180 The text being graded against.
1181
1182 - `JsonValue; type "text_similarity"constant`
1183
1184 The type of grader.
1185
1186 - `TEXT_SIMILARITY("text_similarity")`
1187
1188 - `class PythonGrader:`
1189
1190 A PythonGrader object that runs a python script on the input.
1191
1192 - `String name`
1193
1194 The name of the grader.
1195
1196 - `String source`
1197
1198 The source code of the python script.
1199
1200 - `JsonValue; type "python"constant`
1201
1202 The object type, which is always `python`.
1203
1204 - `PYTHON("python")`
1205
1206 - `Optional<String> imageTag`
1207
1208 The image tag to use for the python script.
1209
1210 - `class ScoreModelGrader:`
1211
1212 A ScoreModelGrader object that uses a model to assign a score to the input.
1213
1214 - `List<Input> input`
1215
1216 The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.
1217
1218 - `Content content`
1219
1220 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.
1221
1222 - `String`
1223
1224 - `class ResponseInputText:`
1225
1226 A text input to the model.
1227
1228 - `String text`
1229
1230 The text input to the model.
1231
1232 - `JsonValue; type "input_text"constant`
1233
1234 The type of the input item. Always `input_text`.
1235
1236 - `INPUT_TEXT("input_text")`
1237
1238 - `class OutputText:`
1239
1240 A text output from the model.
1241
1242 - `String text`
1243
1244 The text output from the model.
1245
1246 - `JsonValue; type "output_text"constant`
1247
1248 The type of the output text. Always `output_text`.
1249
1250 - `OUTPUT_TEXT("output_text")`
1251
1252 - `class InputImage:`
1253
1254 An image input block used within EvalItem content arrays.
1255
1256 - `String imageUrl`
1257
1258 The URL of the image input.
1259
1260 - `JsonValue; type "input_image"constant`
1261
1262 The type of the image input. Always `input_image`.
1263
1264 - `INPUT_IMAGE("input_image")`
1265
1266 - `Optional<String> detail`
1267
1268 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
1269
1270 - `class ResponseInputAudio:`
1271
1272 An audio input to the model.
1273
1274 - `InputAudio inputAudio`
1275
1276 - `String data`
1277
1278 Base64-encoded audio data.
1279
1280 - `Format format`
1281
1282 The format of the audio data. Currently supported formats are `mp3` and
1283 `wav`.
1284
1285 - `MP3("mp3")`
1286
1287 - `WAV("wav")`
1288
1289 - `JsonValue; type "input_audio"constant`
1290
1291 The type of the input item. Always `input_audio`.
1292
1293 - `INPUT_AUDIO("input_audio")`
1294
1295 - `List<EvalContentItem>`
1296
1297 - `String`
1298
1299 - `class ResponseInputText:`
1300
1301 A text input to the model.
1302
1303 - `OutputText`
1304
1305 - `String text`
1306
1307 The text output from the model.
1308
1309 - `JsonValue; type "output_text"constant`
1310
1311 The type of the output text. Always `output_text`.
1312
1313 - `OUTPUT_TEXT("output_text")`
1314
1315 - `InputImage`
1316
1317 - `String imageUrl`
1318
1319 The URL of the image input.
1320
1321 - `JsonValue; type "input_image"constant`
1322
1323 The type of the image input. Always `input_image`.
1324
1325 - `INPUT_IMAGE("input_image")`
1326
1327 - `Optional<String> detail`
1328
1329 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
1330
1331 - `class ResponseInputAudio:`
1332
1333 An audio input to the model.
1334
1335 - `Role role`
1336
1337 The role of the message input. One of `user`, `assistant`, `system`, or
1338 `developer`.
1339
1340 - `USER("user")`
1341
1342 - `ASSISTANT("assistant")`
1343
1344 - `SYSTEM("system")`
1345
1346 - `DEVELOPER("developer")`
1347
1348 - `Optional<Type> type`
1349
1350 The type of the message input. Always `message`.
1351
1352 - `MESSAGE("message")`
1353
1354 - `String model`
1355
1356 The model to use for the evaluation.
1357
1358 - `String name`
1359
1360 The name of the grader.
1361
1362 - `JsonValue; type "score_model"constant`
1363
1364 The object type, which is always `score_model`.
1365
1366 - `SCORE_MODEL("score_model")`
1367
1368 - `Optional<List<Double>> range`
1369
1370 The range of the score. Defaults to `[0, 1]`.
1371
1372 - `Optional<SamplingParams> samplingParams`
1373
1374 The sampling parameters for the model.
1375
1376 - `Optional<Long> maxCompletionsTokens`
1377
1378 The maximum number of tokens the grader model may generate in its response.
1379
1380 - `Optional<ReasoningEffort> reasoningEffort`
1381
1382 Constrains effort on reasoning for
1383 [reasoning models](https://platform.openai.com/docs/guides/reasoning).
1384 Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing
1385 reasoning effort can result in faster responses and fewer tokens used
1386 on reasoning in a response.
1387
1388 - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1.
1389 - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`.
1390 - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort.
1391 - `xhigh` is supported for all models after `gpt-5.1-codex-max`.
1392
1393 - `NONE("none")`
1394
1395 - `MINIMAL("minimal")`
1396
1397 - `LOW("low")`
1398
1399 - `MEDIUM("medium")`
1400
1401 - `HIGH("high")`
1402
1403 - `XHIGH("xhigh")`
1404
1405 - `Optional<Long> seed`
1406
1407 A seed value to initialize the randomness, during sampling.
1408
1409 - `Optional<Double> temperature`
1410
1411 A higher temperature increases randomness in the outputs.
1412
1413 - `Optional<Double> topP`
1414
1415 An alternative to temperature for nucleus sampling; 1.0 includes all tokens.
1416
1417 - `class MultiGrader:`
1418
1419 A MultiGrader object combines the output of multiple graders to produce a single score.
1420
1421 - `String calculateOutput`
1422
1423 A formula to calculate the output based on grader results.
1424
1425 - `Graders graders`
1426
1427 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
1428
1429 - `class StringCheckGrader:`
1430
1431 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
1432
1433 - `class TextSimilarityGrader:`
1434
1435 A TextSimilarityGrader object which grades text based on similarity metrics.
1436
1437 - `class PythonGrader:`
1438
1439 A PythonGrader object that runs a python script on the input.
1440
1441 - `class ScoreModelGrader:`
1442
1443 A ScoreModelGrader object that uses a model to assign a score to the input.
1444
1445 - `class LabelModelGrader:`
1446
1447 A LabelModelGrader object which uses a model to assign labels to each item
1448 in the evaluation.
1449
1450 - `List<Input> input`
1451
1452 - `Content content`
1453
1454 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.
1455
1456 - `String`
1457
1458 - `class ResponseInputText:`
1459
1460 A text input to the model.
1461
1462 - `class OutputText:`
1463
1464 A text output from the model.
1465
1466 - `String text`
1467
1468 The text output from the model.
1469
1470 - `JsonValue; type "output_text"constant`
1471
1472 The type of the output text. Always `output_text`.
1473
1474 - `OUTPUT_TEXT("output_text")`
1475
1476 - `class InputImage:`
1477
1478 An image input block used within EvalItem content arrays.
1479
1480 - `String imageUrl`
1481
1482 The URL of the image input.
1483
1484 - `JsonValue; type "input_image"constant`
1485
1486 The type of the image input. Always `input_image`.
1487
1488 - `INPUT_IMAGE("input_image")`
1489
1490 - `Optional<String> detail`
1491
1492 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
1493
1494 - `class ResponseInputAudio:`
1495
1496 An audio input to the model.
1497
1498 - `List<EvalContentItem>`
1499
1500 - `String`
1501
1502 - `class ResponseInputText:`
1503
1504 A text input to the model.
1505
1506 - `OutputText`
1507
1508 - `InputImage`
1509
1510 - `class ResponseInputAudio:`
1511
1512 An audio input to the model.
1513
1514 - `Role role`
1515
1516 The role of the message input. One of `user`, `assistant`, `system`, or
1517 `developer`.
1518
1519 - `USER("user")`
1520
1521 - `ASSISTANT("assistant")`
1522
1523 - `SYSTEM("system")`
1524
1525 - `DEVELOPER("developer")`
1526
1527 - `Optional<Type> type`
1528
1529 The type of the message input. Always `message`.
1530
1531 - `MESSAGE("message")`
1532
1533 - `List<String> labels`
1534
1535 The labels to assign to each item in the evaluation.
1536
1537 - `String model`
1538
1539 The model to use for the evaluation. Must support structured outputs.
1540
1541 - `String name`
1542
1543 The name of the grader.
1544
1545 - `List<String> passingLabels`
1546
1547 The labels that indicate a passing result. Must be a subset of labels.
1548
1549 - `JsonValue; type "label_model"constant`
1550
1551 The object type, which is always `label_model`.
1552
1553 - `LABEL_MODEL("label_model")`
1554
1555 - `String name`
1556
1557 The name of the grader.
1558
1559 - `JsonValue; type "multi"constant`
1560
1561 The object type, which is always `multi`.
1562
1563 - `MULTI("multi")`
1564
1565### Example
1566
1567```java
1568package com.openai.example;
1569
1570import com.openai.client.OpenAIClient;
1571import com.openai.client.okhttp.OpenAIOkHttpClient;
1572import com.openai.models.finetuning.alpha.graders.GraderValidateParams;
1573import com.openai.models.finetuning.alpha.graders.GraderValidateResponse;
1574import com.openai.models.graders.gradermodels.StringCheckGrader;
1575
1576public final class Main {
1577 private Main() {}
1578
1579 public static void main(String[] args) {
1580 OpenAIClient client = OpenAIOkHttpClient.fromEnv();
1581
1582 GraderValidateParams params = GraderValidateParams.builder()
1583 .grader(StringCheckGrader.builder()
1584 .input("input")
1585 .name("name")
1586 .operation(StringCheckGrader.Operation.EQ)
1587 .reference("reference")
1588 .build())
1589 .build();
1590 GraderValidateResponse response = client.fineTuning().alpha().graders().validate(params);
1591 }
1592}
1593```
1594
1595#### Response
1596
1597```json
1598{
1599 "grader": {
1600 "input": "input",
1601 "name": "name",
1602 "operation": "eq",
1603 "reference": "reference",
1604 "type": "string_check"
1605 }
1606}
1607```