go/resources/fine_tuning/subresources/alpha/index.md +0 −1603 deleted
File Deleted View Diff
1# Alpha
2
3# Graders
4
5## Run grader
6
7`client.FineTuning.Alpha.Graders.Run(ctx, body) (*FineTuningAlphaGraderRunResponse, error)`
8
9**post** `/fine_tuning/alpha/graders/run`
10
11Run a grader.
12
13### Parameters
14
15- `body FineTuningAlphaGraderRunParams`
16
17 - `Grader param.Field[FineTuningAlphaGraderRunParamsGraderUnion]`
18
19 The grader used for the fine-tuning job.
20
21 - `type StringCheckGrader struct{…}`
22
23 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
24
25 - `Input string`
26
27 The input text. This may include template strings.
28
29 - `Name string`
30
31 The name of the grader.
32
33 - `Operation StringCheckGraderOperation`
34
35 The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
36
37 - `const StringCheckGraderOperationEq StringCheckGraderOperation = "eq"`
38
39 - `const StringCheckGraderOperationNe StringCheckGraderOperation = "ne"`
40
41 - `const StringCheckGraderOperationLike StringCheckGraderOperation = "like"`
42
43 - `const StringCheckGraderOperationIlike StringCheckGraderOperation = "ilike"`
44
45 - `Reference string`
46
47 The reference text. This may include template strings.
48
49 - `Type StringCheck`
50
51 The object type, which is always `string_check`.
52
53 - `const StringCheckStringCheck StringCheck = "string_check"`
54
55 - `type TextSimilarityGrader struct{…}`
56
57 A TextSimilarityGrader object which grades text based on similarity metrics.
58
59 - `EvaluationMetric TextSimilarityGraderEvaluationMetric`
60
61 The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`,
62 `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`,
63 or `rouge_l`.
64
65 - `const TextSimilarityGraderEvaluationMetricCosine TextSimilarityGraderEvaluationMetric = "cosine"`
66
67 - `const TextSimilarityGraderEvaluationMetricFuzzyMatch TextSimilarityGraderEvaluationMetric = "fuzzy_match"`
68
69 - `const TextSimilarityGraderEvaluationMetricBleu TextSimilarityGraderEvaluationMetric = "bleu"`
70
71 - `const TextSimilarityGraderEvaluationMetricGleu TextSimilarityGraderEvaluationMetric = "gleu"`
72
73 - `const TextSimilarityGraderEvaluationMetricMeteor TextSimilarityGraderEvaluationMetric = "meteor"`
74
75 - `const TextSimilarityGraderEvaluationMetricRouge1 TextSimilarityGraderEvaluationMetric = "rouge_1"`
76
77 - `const TextSimilarityGraderEvaluationMetricRouge2 TextSimilarityGraderEvaluationMetric = "rouge_2"`
78
79 - `const TextSimilarityGraderEvaluationMetricRouge3 TextSimilarityGraderEvaluationMetric = "rouge_3"`
80
81 - `const TextSimilarityGraderEvaluationMetricRouge4 TextSimilarityGraderEvaluationMetric = "rouge_4"`
82
83 - `const TextSimilarityGraderEvaluationMetricRouge5 TextSimilarityGraderEvaluationMetric = "rouge_5"`
84
85 - `const TextSimilarityGraderEvaluationMetricRougeL TextSimilarityGraderEvaluationMetric = "rouge_l"`
86
87 - `Input string`
88
89 The text being graded.
90
91 - `Name string`
92
93 The name of the grader.
94
95 - `Reference string`
96
97 The text being graded against.
98
99 - `Type TextSimilarity`
100
101 The type of grader.
102
103 - `const TextSimilarityTextSimilarity TextSimilarity = "text_similarity"`
104
105 - `type PythonGrader struct{…}`
106
107 A PythonGrader object that runs a python script on the input.
108
109 - `Name string`
110
111 The name of the grader.
112
113 - `Source string`
114
115 The source code of the python script.
116
117 - `Type Python`
118
119 The object type, which is always `python`.
120
121 - `const PythonPython Python = "python"`
122
123 - `ImageTag string`
124
125 The image tag to use for the python script.
126
127 - `type ScoreModelGrader struct{…}`
128
129 A ScoreModelGrader object that uses a model to assign a score to the input.
130
131 - `Input []ScoreModelGraderInput`
132
133 The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.
134
135 - `Content ScoreModelGraderInputContentUnion`
136
137 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.
138
139 - `string`
140
141 - `type ResponseInputText struct{…}`
142
143 A text input to the model.
144
145 - `Text string`
146
147 The text input to the model.
148
149 - `Type InputText`
150
151 The type of the input item. Always `input_text`.
152
153 - `const InputTextInputText InputText = "input_text"`
154
155 - `type ScoreModelGraderInputContentOutputText struct{…}`
156
157 A text output from the model.
158
159 - `Text string`
160
161 The text output from the model.
162
163 - `Type OutputText`
164
165 The type of the output text. Always `output_text`.
166
167 - `const OutputTextOutputText OutputText = "output_text"`
168
169 - `type ScoreModelGraderInputContentInputImage struct{…}`
170
171 An image input block used within EvalItem content arrays.
172
173 - `ImageURL string`
174
175 The URL of the image input.
176
177 - `Type InputImage`
178
179 The type of the image input. Always `input_image`.
180
181 - `const InputImageInputImage InputImage = "input_image"`
182
183 - `Detail string`
184
185 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
186
187 - `type ResponseInputAudio struct{…}`
188
189 An audio input to the model.
190
191 - `InputAudio ResponseInputAudioInputAudio`
192
193 - `Data string`
194
195 Base64-encoded audio data.
196
197 - `Format string`
198
199 The format of the audio data. Currently supported formats are `mp3` and
200 `wav`.
201
202 - `const ResponseInputAudioInputAudioFormatMP3 ResponseInputAudioInputAudioFormat = "mp3"`
203
204 - `const ResponseInputAudioInputAudioFormatWAV ResponseInputAudioInputAudioFormat = "wav"`
205
206 - `Type InputAudio`
207
208 The type of the input item. Always `input_audio`.
209
210 - `const InputAudioInputAudio InputAudio = "input_audio"`
211
212 - `type GraderInputs []GraderInputUnion`
213
214 A list of inputs, each of which may be either an input text, output text, input
215 image, or input audio object.
216
217 - `string`
218
219 - `type ResponseInputText struct{…}`
220
221 A text input to the model.
222
223 - `type GraderInputOutputText struct{…}`
224
225 A text output from the model.
226
227 - `Text string`
228
229 The text output from the model.
230
231 - `Type OutputText`
232
233 The type of the output text. Always `output_text`.
234
235 - `const OutputTextOutputText OutputText = "output_text"`
236
237 - `type GraderInputInputImage struct{…}`
238
239 An image input block used within EvalItem content arrays.
240
241 - `ImageURL string`
242
243 The URL of the image input.
244
245 - `Type InputImage`
246
247 The type of the image input. Always `input_image`.
248
249 - `const InputImageInputImage InputImage = "input_image"`
250
251 - `Detail string`
252
253 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
254
255 - `type ResponseInputAudio struct{…}`
256
257 An audio input to the model.
258
259 - `Role string`
260
261 The role of the message input. One of `user`, `assistant`, `system`, or
262 `developer`.
263
264 - `const ScoreModelGraderInputRoleUser ScoreModelGraderInputRole = "user"`
265
266 - `const ScoreModelGraderInputRoleAssistant ScoreModelGraderInputRole = "assistant"`
267
268 - `const ScoreModelGraderInputRoleSystem ScoreModelGraderInputRole = "system"`
269
270 - `const ScoreModelGraderInputRoleDeveloper ScoreModelGraderInputRole = "developer"`
271
272 - `Type string`
273
274 The type of the message input. Always `message`.
275
276 - `const ScoreModelGraderInputTypeMessage ScoreModelGraderInputType = "message"`
277
278 - `Model string`
279
280 The model to use for the evaluation.
281
282 - `Name string`
283
284 The name of the grader.
285
286 - `Type ScoreModel`
287
288 The object type, which is always `score_model`.
289
290 - `const ScoreModelScoreModel ScoreModel = "score_model"`
291
292 - `Range []float64`
293
294 The range of the score. Defaults to `[0, 1]`.
295
296 - `SamplingParams ScoreModelGraderSamplingParams`
297
298 The sampling parameters for the model.
299
300 - `MaxCompletionsTokens int64`
301
302 The maximum number of tokens the grader model may generate in its response.
303
304 - `ReasoningEffort ReasoningEffort`
305
306 Constrains effort on reasoning for
307 [reasoning models](https://platform.openai.com/docs/guides/reasoning).
308 Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing
309 reasoning effort can result in faster responses and fewer tokens used
310 on reasoning in a response.
311
312 - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1.
313 - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`.
314 - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort.
315 - `xhigh` is supported for all models after `gpt-5.1-codex-max`.
316
317 - `const ReasoningEffortNone ReasoningEffort = "none"`
318
319 - `const ReasoningEffortMinimal ReasoningEffort = "minimal"`
320
321 - `const ReasoningEffortLow ReasoningEffort = "low"`
322
323 - `const ReasoningEffortMedium ReasoningEffort = "medium"`
324
325 - `const ReasoningEffortHigh ReasoningEffort = "high"`
326
327 - `const ReasoningEffortXhigh ReasoningEffort = "xhigh"`
328
329 - `Seed int64`
330
331 A seed value to initialize the randomness, during sampling.
332
333 - `Temperature float64`
334
335 A higher temperature increases randomness in the outputs.
336
337 - `TopP float64`
338
339 An alternative to temperature for nucleus sampling; 1.0 includes all tokens.
340
341 - `type MultiGrader struct{…}`
342
343 A MultiGrader object combines the output of multiple graders to produce a single score.
344
345 - `CalculateOutput string`
346
347 A formula to calculate the output based on grader results.
348
349 - `Graders MultiGraderGradersUnion`
350
351 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
352
353 - `type StringCheckGrader struct{…}`
354
355 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
356
357 - `type TextSimilarityGrader struct{…}`
358
359 A TextSimilarityGrader object which grades text based on similarity metrics.
360
361 - `type PythonGrader struct{…}`
362
363 A PythonGrader object that runs a python script on the input.
364
365 - `type ScoreModelGrader struct{…}`
366
367 A ScoreModelGrader object that uses a model to assign a score to the input.
368
369 - `type LabelModelGrader struct{…}`
370
371 A LabelModelGrader object which uses a model to assign labels to each item
372 in the evaluation.
373
374 - `Input []LabelModelGraderInput`
375
376 - `Content LabelModelGraderInputContentUnion`
377
378 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.
379
380 - `string`
381
382 - `type ResponseInputText struct{…}`
383
384 A text input to the model.
385
386 - `type LabelModelGraderInputContentOutputText struct{…}`
387
388 A text output from the model.
389
390 - `Text string`
391
392 The text output from the model.
393
394 - `Type OutputText`
395
396 The type of the output text. Always `output_text`.
397
398 - `const OutputTextOutputText OutputText = "output_text"`
399
400 - `type LabelModelGraderInputContentInputImage struct{…}`
401
402 An image input block used within EvalItem content arrays.
403
404 - `ImageURL string`
405
406 The URL of the image input.
407
408 - `Type InputImage`
409
410 The type of the image input. Always `input_image`.
411
412 - `const InputImageInputImage InputImage = "input_image"`
413
414 - `Detail string`
415
416 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
417
418 - `type ResponseInputAudio struct{…}`
419
420 An audio input to the model.
421
422 - `type GraderInputs []GraderInputUnion`
423
424 A list of inputs, each of which may be either an input text, output text, input
425 image, or input audio object.
426
427 - `Role string`
428
429 The role of the message input. One of `user`, `assistant`, `system`, or
430 `developer`.
431
432 - `const LabelModelGraderInputRoleUser LabelModelGraderInputRole = "user"`
433
434 - `const LabelModelGraderInputRoleAssistant LabelModelGraderInputRole = "assistant"`
435
436 - `const LabelModelGraderInputRoleSystem LabelModelGraderInputRole = "system"`
437
438 - `const LabelModelGraderInputRoleDeveloper LabelModelGraderInputRole = "developer"`
439
440 - `Type string`
441
442 The type of the message input. Always `message`.
443
444 - `const LabelModelGraderInputTypeMessage LabelModelGraderInputType = "message"`
445
446 - `Labels []string`
447
448 The labels to assign to each item in the evaluation.
449
450 - `Model string`
451
452 The model to use for the evaluation. Must support structured outputs.
453
454 - `Name string`
455
456 The name of the grader.
457
458 - `PassingLabels []string`
459
460 The labels that indicate a passing result. Must be a subset of labels.
461
462 - `Type LabelModel`
463
464 The object type, which is always `label_model`.
465
466 - `const LabelModelLabelModel LabelModel = "label_model"`
467
468 - `Name string`
469
470 The name of the grader.
471
472 - `Type Multi`
473
474 The object type, which is always `multi`.
475
476 - `const MultiMulti Multi = "multi"`
477
478 - `ModelSample param.Field[string]`
479
480 The model sample to be evaluated. This value will be used to populate
481 the `sample` namespace. See [the guide](https://platform.openai.com/docs/guides/graders) for more details.
482 The `output_json` variable will be populated if the model sample is a
483 valid JSON string.
484
485 - `Item param.Field[any]`
486
487 The dataset item provided to the grader. This will be used to populate
488 the `item` namespace. See [the guide](https://platform.openai.com/docs/guides/graders) for more details.
489
490### Returns
491
492- `type FineTuningAlphaGraderRunResponse struct{…}`
493
494 - `Metadata FineTuningAlphaGraderRunResponseMetadata`
495
496 - `Errors FineTuningAlphaGraderRunResponseMetadataErrors`
497
498 - `FormulaParseError bool`
499
500 - `InvalidVariableError bool`
501
502 - `ModelGraderParseError bool`
503
504 - `ModelGraderRefusalError bool`
505
506 - `ModelGraderServerError bool`
507
508 - `ModelGraderServerErrorDetails string`
509
510 - `OtherError bool`
511
512 - `PythonGraderRuntimeError bool`
513
514 - `PythonGraderRuntimeErrorDetails string`
515
516 - `PythonGraderServerError bool`
517
518 - `PythonGraderServerErrorType string`
519
520 - `SampleParseError bool`
521
522 - `TruncatedObservationError bool`
523
524 - `UnresponsiveRewardError bool`
525
526 - `ExecutionTime float64`
527
528 - `Name string`
529
530 - `SampledModelName string`
531
532 - `Scores map[string, any]`
533
534 - `TokenUsage int64`
535
536 - `Type string`
537
538 - `ModelGraderTokenUsagePerModel map[string, any]`
539
540 - `Reward float64`
541
542 - `SubRewards map[string, any]`
543
544### Example
545
546```go
547package main
548
549import (
550 "context"
551 "fmt"
552
553 "github.com/openai/openai-go"
554 "github.com/openai/openai-go/option"
555)
556
557func main() {
558 client := openai.NewClient(
559 option.WithAPIKey("My API Key"),
560 )
561 response, err := client.FineTuning.Alpha.Graders.Run(context.TODO(), openai.FineTuningAlphaGraderRunParams{
562 Grader: openai.FineTuningAlphaGraderRunParamsGraderUnion{
563 OfStringCheck: &openai.StringCheckGraderParam{
564 Input: "input",
565 Name: "name",
566 Operation: openai.StringCheckGraderOperationEq,
567 Reference: "reference",
568 },
569 },
570 ModelSample: "model_sample",
571 })
572 if err != nil {
573 panic(err.Error())
574 }
575 fmt.Printf("%+v\n", response.Metadata)
576}
577```
578
579#### Response
580
581```json
582{
583 "metadata": {
584 "errors": {
585 "formula_parse_error": true,
586 "invalid_variable_error": true,
587 "model_grader_parse_error": true,
588 "model_grader_refusal_error": true,
589 "model_grader_server_error": true,
590 "model_grader_server_error_details": "model_grader_server_error_details",
591 "other_error": true,
592 "python_grader_runtime_error": true,
593 "python_grader_runtime_error_details": "python_grader_runtime_error_details",
594 "python_grader_server_error": true,
595 "python_grader_server_error_type": "python_grader_server_error_type",
596 "sample_parse_error": true,
597 "truncated_observation_error": true,
598 "unresponsive_reward_error": true
599 },
600 "execution_time": 0,
601 "name": "name",
602 "sampled_model_name": "sampled_model_name",
603 "scores": {
604 "foo": "bar"
605 },
606 "token_usage": 0,
607 "type": "type"
608 },
609 "model_grader_token_usage_per_model": {
610 "foo": "bar"
611 },
612 "reward": 0,
613 "sub_rewards": {
614 "foo": "bar"
615 }
616}
617```
618
619## Validate grader
620
621`client.FineTuning.Alpha.Graders.Validate(ctx, body) (*FineTuningAlphaGraderValidateResponse, error)`
622
623**post** `/fine_tuning/alpha/graders/validate`
624
625Validate a grader.
626
627### Parameters
628
629- `body FineTuningAlphaGraderValidateParams`
630
631 - `Grader param.Field[FineTuningAlphaGraderValidateParamsGraderUnion]`
632
633 The grader used for the fine-tuning job.
634
635 - `type StringCheckGrader struct{…}`
636
637 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
638
639 - `Input string`
640
641 The input text. This may include template strings.
642
643 - `Name string`
644
645 The name of the grader.
646
647 - `Operation StringCheckGraderOperation`
648
649 The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
650
651 - `const StringCheckGraderOperationEq StringCheckGraderOperation = "eq"`
652
653 - `const StringCheckGraderOperationNe StringCheckGraderOperation = "ne"`
654
655 - `const StringCheckGraderOperationLike StringCheckGraderOperation = "like"`
656
657 - `const StringCheckGraderOperationIlike StringCheckGraderOperation = "ilike"`
658
659 - `Reference string`
660
661 The reference text. This may include template strings.
662
663 - `Type StringCheck`
664
665 The object type, which is always `string_check`.
666
667 - `const StringCheckStringCheck StringCheck = "string_check"`
668
669 - `type TextSimilarityGrader struct{…}`
670
671 A TextSimilarityGrader object which grades text based on similarity metrics.
672
673 - `EvaluationMetric TextSimilarityGraderEvaluationMetric`
674
675 The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`,
676 `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`,
677 or `rouge_l`.
678
679 - `const TextSimilarityGraderEvaluationMetricCosine TextSimilarityGraderEvaluationMetric = "cosine"`
680
681 - `const TextSimilarityGraderEvaluationMetricFuzzyMatch TextSimilarityGraderEvaluationMetric = "fuzzy_match"`
682
683 - `const TextSimilarityGraderEvaluationMetricBleu TextSimilarityGraderEvaluationMetric = "bleu"`
684
685 - `const TextSimilarityGraderEvaluationMetricGleu TextSimilarityGraderEvaluationMetric = "gleu"`
686
687 - `const TextSimilarityGraderEvaluationMetricMeteor TextSimilarityGraderEvaluationMetric = "meteor"`
688
689 - `const TextSimilarityGraderEvaluationMetricRouge1 TextSimilarityGraderEvaluationMetric = "rouge_1"`
690
691 - `const TextSimilarityGraderEvaluationMetricRouge2 TextSimilarityGraderEvaluationMetric = "rouge_2"`
692
693 - `const TextSimilarityGraderEvaluationMetricRouge3 TextSimilarityGraderEvaluationMetric = "rouge_3"`
694
695 - `const TextSimilarityGraderEvaluationMetricRouge4 TextSimilarityGraderEvaluationMetric = "rouge_4"`
696
697 - `const TextSimilarityGraderEvaluationMetricRouge5 TextSimilarityGraderEvaluationMetric = "rouge_5"`
698
699 - `const TextSimilarityGraderEvaluationMetricRougeL TextSimilarityGraderEvaluationMetric = "rouge_l"`
700
701 - `Input string`
702
703 The text being graded.
704
705 - `Name string`
706
707 The name of the grader.
708
709 - `Reference string`
710
711 The text being graded against.
712
713 - `Type TextSimilarity`
714
715 The type of grader.
716
717 - `const TextSimilarityTextSimilarity TextSimilarity = "text_similarity"`
718
719 - `type PythonGrader struct{…}`
720
721 A PythonGrader object that runs a python script on the input.
722
723 - `Name string`
724
725 The name of the grader.
726
727 - `Source string`
728
729 The source code of the python script.
730
731 - `Type Python`
732
733 The object type, which is always `python`.
734
735 - `const PythonPython Python = "python"`
736
737 - `ImageTag string`
738
739 The image tag to use for the python script.
740
741 - `type ScoreModelGrader struct{…}`
742
743 A ScoreModelGrader object that uses a model to assign a score to the input.
744
745 - `Input []ScoreModelGraderInput`
746
747 The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.
748
749 - `Content ScoreModelGraderInputContentUnion`
750
751 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.
752
753 - `string`
754
755 - `type ResponseInputText struct{…}`
756
757 A text input to the model.
758
759 - `Text string`
760
761 The text input to the model.
762
763 - `Type InputText`
764
765 The type of the input item. Always `input_text`.
766
767 - `const InputTextInputText InputText = "input_text"`
768
769 - `type ScoreModelGraderInputContentOutputText struct{…}`
770
771 A text output from the model.
772
773 - `Text string`
774
775 The text output from the model.
776
777 - `Type OutputText`
778
779 The type of the output text. Always `output_text`.
780
781 - `const OutputTextOutputText OutputText = "output_text"`
782
783 - `type ScoreModelGraderInputContentInputImage struct{…}`
784
785 An image input block used within EvalItem content arrays.
786
787 - `ImageURL string`
788
789 The URL of the image input.
790
791 - `Type InputImage`
792
793 The type of the image input. Always `input_image`.
794
795 - `const InputImageInputImage InputImage = "input_image"`
796
797 - `Detail string`
798
799 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
800
801 - `type ResponseInputAudio struct{…}`
802
803 An audio input to the model.
804
805 - `InputAudio ResponseInputAudioInputAudio`
806
807 - `Data string`
808
809 Base64-encoded audio data.
810
811 - `Format string`
812
813 The format of the audio data. Currently supported formats are `mp3` and
814 `wav`.
815
816 - `const ResponseInputAudioInputAudioFormatMP3 ResponseInputAudioInputAudioFormat = "mp3"`
817
818 - `const ResponseInputAudioInputAudioFormatWAV ResponseInputAudioInputAudioFormat = "wav"`
819
820 - `Type InputAudio`
821
822 The type of the input item. Always `input_audio`.
823
824 - `const InputAudioInputAudio InputAudio = "input_audio"`
825
826 - `type GraderInputs []GraderInputUnion`
827
828 A list of inputs, each of which may be either an input text, output text, input
829 image, or input audio object.
830
831 - `string`
832
833 - `type ResponseInputText struct{…}`
834
835 A text input to the model.
836
837 - `type GraderInputOutputText struct{…}`
838
839 A text output from the model.
840
841 - `Text string`
842
843 The text output from the model.
844
845 - `Type OutputText`
846
847 The type of the output text. Always `output_text`.
848
849 - `const OutputTextOutputText OutputText = "output_text"`
850
851 - `type GraderInputInputImage struct{…}`
852
853 An image input block used within EvalItem content arrays.
854
855 - `ImageURL string`
856
857 The URL of the image input.
858
859 - `Type InputImage`
860
861 The type of the image input. Always `input_image`.
862
863 - `const InputImageInputImage InputImage = "input_image"`
864
865 - `Detail string`
866
867 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
868
869 - `type ResponseInputAudio struct{…}`
870
871 An audio input to the model.
872
873 - `Role string`
874
875 The role of the message input. One of `user`, `assistant`, `system`, or
876 `developer`.
877
878 - `const ScoreModelGraderInputRoleUser ScoreModelGraderInputRole = "user"`
879
880 - `const ScoreModelGraderInputRoleAssistant ScoreModelGraderInputRole = "assistant"`
881
882 - `const ScoreModelGraderInputRoleSystem ScoreModelGraderInputRole = "system"`
883
884 - `const ScoreModelGraderInputRoleDeveloper ScoreModelGraderInputRole = "developer"`
885
886 - `Type string`
887
888 The type of the message input. Always `message`.
889
890 - `const ScoreModelGraderInputTypeMessage ScoreModelGraderInputType = "message"`
891
892 - `Model string`
893
894 The model to use for the evaluation.
895
896 - `Name string`
897
898 The name of the grader.
899
900 - `Type ScoreModel`
901
902 The object type, which is always `score_model`.
903
904 - `const ScoreModelScoreModel ScoreModel = "score_model"`
905
906 - `Range []float64`
907
908 The range of the score. Defaults to `[0, 1]`.
909
910 - `SamplingParams ScoreModelGraderSamplingParams`
911
912 The sampling parameters for the model.
913
914 - `MaxCompletionsTokens int64`
915
916 The maximum number of tokens the grader model may generate in its response.
917
918 - `ReasoningEffort ReasoningEffort`
919
920 Constrains effort on reasoning for
921 [reasoning models](https://platform.openai.com/docs/guides/reasoning).
922 Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing
923 reasoning effort can result in faster responses and fewer tokens used
924 on reasoning in a response.
925
926 - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1.
927 - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`.
928 - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort.
929 - `xhigh` is supported for all models after `gpt-5.1-codex-max`.
930
931 - `const ReasoningEffortNone ReasoningEffort = "none"`
932
933 - `const ReasoningEffortMinimal ReasoningEffort = "minimal"`
934
935 - `const ReasoningEffortLow ReasoningEffort = "low"`
936
937 - `const ReasoningEffortMedium ReasoningEffort = "medium"`
938
939 - `const ReasoningEffortHigh ReasoningEffort = "high"`
940
941 - `const ReasoningEffortXhigh ReasoningEffort = "xhigh"`
942
943 - `Seed int64`
944
945 A seed value to initialize the randomness, during sampling.
946
947 - `Temperature float64`
948
949 A higher temperature increases randomness in the outputs.
950
951 - `TopP float64`
952
953 An alternative to temperature for nucleus sampling; 1.0 includes all tokens.
954
955 - `type MultiGrader struct{…}`
956
957 A MultiGrader object combines the output of multiple graders to produce a single score.
958
959 - `CalculateOutput string`
960
961 A formula to calculate the output based on grader results.
962
963 - `Graders MultiGraderGradersUnion`
964
965 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
966
967 - `type StringCheckGrader struct{…}`
968
969 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
970
971 - `type TextSimilarityGrader struct{…}`
972
973 A TextSimilarityGrader object which grades text based on similarity metrics.
974
975 - `type PythonGrader struct{…}`
976
977 A PythonGrader object that runs a python script on the input.
978
979 - `type ScoreModelGrader struct{…}`
980
981 A ScoreModelGrader object that uses a model to assign a score to the input.
982
983 - `type LabelModelGrader struct{…}`
984
985 A LabelModelGrader object which uses a model to assign labels to each item
986 in the evaluation.
987
988 - `Input []LabelModelGraderInput`
989
990 - `Content LabelModelGraderInputContentUnion`
991
992 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.
993
994 - `string`
995
996 - `type ResponseInputText struct{…}`
997
998 A text input to the model.
999
1000 - `type LabelModelGraderInputContentOutputText struct{…}`
1001
1002 A text output from the model.
1003
1004 - `Text string`
1005
1006 The text output from the model.
1007
1008 - `Type OutputText`
1009
1010 The type of the output text. Always `output_text`.
1011
1012 - `const OutputTextOutputText OutputText = "output_text"`
1013
1014 - `type LabelModelGraderInputContentInputImage struct{…}`
1015
1016 An image input block used within EvalItem content arrays.
1017
1018 - `ImageURL string`
1019
1020 The URL of the image input.
1021
1022 - `Type InputImage`
1023
1024 The type of the image input. Always `input_image`.
1025
1026 - `const InputImageInputImage InputImage = "input_image"`
1027
1028 - `Detail string`
1029
1030 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
1031
1032 - `type ResponseInputAudio struct{…}`
1033
1034 An audio input to the model.
1035
1036 - `type GraderInputs []GraderInputUnion`
1037
1038 A list of inputs, each of which may be either an input text, output text, input
1039 image, or input audio object.
1040
1041 - `Role string`
1042
1043 The role of the message input. One of `user`, `assistant`, `system`, or
1044 `developer`.
1045
1046 - `const LabelModelGraderInputRoleUser LabelModelGraderInputRole = "user"`
1047
1048 - `const LabelModelGraderInputRoleAssistant LabelModelGraderInputRole = "assistant"`
1049
1050 - `const LabelModelGraderInputRoleSystem LabelModelGraderInputRole = "system"`
1051
1052 - `const LabelModelGraderInputRoleDeveloper LabelModelGraderInputRole = "developer"`
1053
1054 - `Type string`
1055
1056 The type of the message input. Always `message`.
1057
1058 - `const LabelModelGraderInputTypeMessage LabelModelGraderInputType = "message"`
1059
1060 - `Labels []string`
1061
1062 The labels to assign to each item in the evaluation.
1063
1064 - `Model string`
1065
1066 The model to use for the evaluation. Must support structured outputs.
1067
1068 - `Name string`
1069
1070 The name of the grader.
1071
1072 - `PassingLabels []string`
1073
1074 The labels that indicate a passing result. Must be a subset of labels.
1075
1076 - `Type LabelModel`
1077
1078 The object type, which is always `label_model`.
1079
1080 - `const LabelModelLabelModel LabelModel = "label_model"`
1081
1082 - `Name string`
1083
1084 The name of the grader.
1085
1086 - `Type Multi`
1087
1088 The object type, which is always `multi`.
1089
1090 - `const MultiMulti Multi = "multi"`
1091
1092### Returns
1093
1094- `type FineTuningAlphaGraderValidateResponse struct{…}`
1095
1096 - `Grader FineTuningAlphaGraderValidateResponseGraderUnion`
1097
1098 The grader used for the fine-tuning job.
1099
1100 - `type StringCheckGrader struct{…}`
1101
1102 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
1103
1104 - `Input string`
1105
1106 The input text. This may include template strings.
1107
1108 - `Name string`
1109
1110 The name of the grader.
1111
1112 - `Operation StringCheckGraderOperation`
1113
1114 The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
1115
1116 - `const StringCheckGraderOperationEq StringCheckGraderOperation = "eq"`
1117
1118 - `const StringCheckGraderOperationNe StringCheckGraderOperation = "ne"`
1119
1120 - `const StringCheckGraderOperationLike StringCheckGraderOperation = "like"`
1121
1122 - `const StringCheckGraderOperationIlike StringCheckGraderOperation = "ilike"`
1123
1124 - `Reference string`
1125
1126 The reference text. This may include template strings.
1127
1128 - `Type StringCheck`
1129
1130 The object type, which is always `string_check`.
1131
1132 - `const StringCheckStringCheck StringCheck = "string_check"`
1133
1134 - `type TextSimilarityGrader struct{…}`
1135
1136 A TextSimilarityGrader object which grades text based on similarity metrics.
1137
1138 - `EvaluationMetric TextSimilarityGraderEvaluationMetric`
1139
1140 The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`,
1141 `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`,
1142 or `rouge_l`.
1143
1144 - `const TextSimilarityGraderEvaluationMetricCosine TextSimilarityGraderEvaluationMetric = "cosine"`
1145
1146 - `const TextSimilarityGraderEvaluationMetricFuzzyMatch TextSimilarityGraderEvaluationMetric = "fuzzy_match"`
1147
1148 - `const TextSimilarityGraderEvaluationMetricBleu TextSimilarityGraderEvaluationMetric = "bleu"`
1149
1150 - `const TextSimilarityGraderEvaluationMetricGleu TextSimilarityGraderEvaluationMetric = "gleu"`
1151
1152 - `const TextSimilarityGraderEvaluationMetricMeteor TextSimilarityGraderEvaluationMetric = "meteor"`
1153
1154 - `const TextSimilarityGraderEvaluationMetricRouge1 TextSimilarityGraderEvaluationMetric = "rouge_1"`
1155
1156 - `const TextSimilarityGraderEvaluationMetricRouge2 TextSimilarityGraderEvaluationMetric = "rouge_2"`
1157
1158 - `const TextSimilarityGraderEvaluationMetricRouge3 TextSimilarityGraderEvaluationMetric = "rouge_3"`
1159
1160 - `const TextSimilarityGraderEvaluationMetricRouge4 TextSimilarityGraderEvaluationMetric = "rouge_4"`
1161
1162 - `const TextSimilarityGraderEvaluationMetricRouge5 TextSimilarityGraderEvaluationMetric = "rouge_5"`
1163
1164 - `const TextSimilarityGraderEvaluationMetricRougeL TextSimilarityGraderEvaluationMetric = "rouge_l"`
1165
1166 - `Input string`
1167
1168 The text being graded.
1169
1170 - `Name string`
1171
1172 The name of the grader.
1173
1174 - `Reference string`
1175
1176 The text being graded against.
1177
1178 - `Type TextSimilarity`
1179
1180 The type of grader.
1181
1182 - `const TextSimilarityTextSimilarity TextSimilarity = "text_similarity"`
1183
1184 - `type PythonGrader struct{…}`
1185
1186 A PythonGrader object that runs a python script on the input.
1187
1188 - `Name string`
1189
1190 The name of the grader.
1191
1192 - `Source string`
1193
1194 The source code of the python script.
1195
1196 - `Type Python`
1197
1198 The object type, which is always `python`.
1199
1200 - `const PythonPython Python = "python"`
1201
1202 - `ImageTag string`
1203
1204 The image tag to use for the python script.
1205
1206 - `type ScoreModelGrader struct{…}`
1207
1208 A ScoreModelGrader object that uses a model to assign a score to the input.
1209
1210 - `Input []ScoreModelGraderInput`
1211
1212 The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.
1213
1214 - `Content ScoreModelGraderInputContentUnion`
1215
1216 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.
1217
1218 - `string`
1219
1220 - `type ResponseInputText struct{…}`
1221
1222 A text input to the model.
1223
1224 - `Text string`
1225
1226 The text input to the model.
1227
1228 - `Type InputText`
1229
1230 The type of the input item. Always `input_text`.
1231
1232 - `const InputTextInputText InputText = "input_text"`
1233
1234 - `type ScoreModelGraderInputContentOutputText struct{…}`
1235
1236 A text output from the model.
1237
1238 - `Text string`
1239
1240 The text output from the model.
1241
1242 - `Type OutputText`
1243
1244 The type of the output text. Always `output_text`.
1245
1246 - `const OutputTextOutputText OutputText = "output_text"`
1247
1248 - `type ScoreModelGraderInputContentInputImage struct{…}`
1249
1250 An image input block used within EvalItem content arrays.
1251
1252 - `ImageURL string`
1253
1254 The URL of the image input.
1255
1256 - `Type InputImage`
1257
1258 The type of the image input. Always `input_image`.
1259
1260 - `const InputImageInputImage InputImage = "input_image"`
1261
1262 - `Detail string`
1263
1264 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
1265
1266 - `type ResponseInputAudio struct{…}`
1267
1268 An audio input to the model.
1269
1270 - `InputAudio ResponseInputAudioInputAudio`
1271
1272 - `Data string`
1273
1274 Base64-encoded audio data.
1275
1276 - `Format string`
1277
1278 The format of the audio data. Currently supported formats are `mp3` and
1279 `wav`.
1280
1281 - `const ResponseInputAudioInputAudioFormatMP3 ResponseInputAudioInputAudioFormat = "mp3"`
1282
1283 - `const ResponseInputAudioInputAudioFormatWAV ResponseInputAudioInputAudioFormat = "wav"`
1284
1285 - `Type InputAudio`
1286
1287 The type of the input item. Always `input_audio`.
1288
1289 - `const InputAudioInputAudio InputAudio = "input_audio"`
1290
1291 - `type GraderInputs []GraderInputUnion`
1292
1293 A list of inputs, each of which may be either an input text, output text, input
1294 image, or input audio object.
1295
1296 - `string`
1297
1298 - `type ResponseInputText struct{…}`
1299
1300 A text input to the model.
1301
1302 - `type GraderInputOutputText struct{…}`
1303
1304 A text output from the model.
1305
1306 - `Text string`
1307
1308 The text output from the model.
1309
1310 - `Type OutputText`
1311
1312 The type of the output text. Always `output_text`.
1313
1314 - `const OutputTextOutputText OutputText = "output_text"`
1315
1316 - `type GraderInputInputImage struct{…}`
1317
1318 An image input block used within EvalItem content arrays.
1319
1320 - `ImageURL string`
1321
1322 The URL of the image input.
1323
1324 - `Type InputImage`
1325
1326 The type of the image input. Always `input_image`.
1327
1328 - `const InputImageInputImage InputImage = "input_image"`
1329
1330 - `Detail string`
1331
1332 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
1333
1334 - `type ResponseInputAudio struct{…}`
1335
1336 An audio input to the model.
1337
1338 - `Role string`
1339
1340 The role of the message input. One of `user`, `assistant`, `system`, or
1341 `developer`.
1342
1343 - `const ScoreModelGraderInputRoleUser ScoreModelGraderInputRole = "user"`
1344
1345 - `const ScoreModelGraderInputRoleAssistant ScoreModelGraderInputRole = "assistant"`
1346
1347 - `const ScoreModelGraderInputRoleSystem ScoreModelGraderInputRole = "system"`
1348
1349 - `const ScoreModelGraderInputRoleDeveloper ScoreModelGraderInputRole = "developer"`
1350
1351 - `Type string`
1352
1353 The type of the message input. Always `message`.
1354
1355 - `const ScoreModelGraderInputTypeMessage ScoreModelGraderInputType = "message"`
1356
1357 - `Model string`
1358
1359 The model to use for the evaluation.
1360
1361 - `Name string`
1362
1363 The name of the grader.
1364
1365 - `Type ScoreModel`
1366
1367 The object type, which is always `score_model`.
1368
1369 - `const ScoreModelScoreModel ScoreModel = "score_model"`
1370
1371 - `Range []float64`
1372
1373 The range of the score. Defaults to `[0, 1]`.
1374
1375 - `SamplingParams ScoreModelGraderSamplingParams`
1376
1377 The sampling parameters for the model.
1378
1379 - `MaxCompletionsTokens int64`
1380
1381 The maximum number of tokens the grader model may generate in its response.
1382
1383 - `ReasoningEffort ReasoningEffort`
1384
1385 Constrains effort on reasoning for
1386 [reasoning models](https://platform.openai.com/docs/guides/reasoning).
1387 Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing
1388 reasoning effort can result in faster responses and fewer tokens used
1389 on reasoning in a response.
1390
1391 - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1.
1392 - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`.
1393 - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort.
1394 - `xhigh` is supported for all models after `gpt-5.1-codex-max`.
1395
1396 - `const ReasoningEffortNone ReasoningEffort = "none"`
1397
1398 - `const ReasoningEffortMinimal ReasoningEffort = "minimal"`
1399
1400 - `const ReasoningEffortLow ReasoningEffort = "low"`
1401
1402 - `const ReasoningEffortMedium ReasoningEffort = "medium"`
1403
1404 - `const ReasoningEffortHigh ReasoningEffort = "high"`
1405
1406 - `const ReasoningEffortXhigh ReasoningEffort = "xhigh"`
1407
1408 - `Seed int64`
1409
1410 A seed value to initialize the randomness, during sampling.
1411
1412 - `Temperature float64`
1413
1414 A higher temperature increases randomness in the outputs.
1415
1416 - `TopP float64`
1417
1418 An alternative to temperature for nucleus sampling; 1.0 includes all tokens.
1419
1420 - `type MultiGrader struct{…}`
1421
1422 A MultiGrader object combines the output of multiple graders to produce a single score.
1423
1424 - `CalculateOutput string`
1425
1426 A formula to calculate the output based on grader results.
1427
1428 - `Graders MultiGraderGradersUnion`
1429
1430 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
1431
1432 - `type StringCheckGrader struct{…}`
1433
1434 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
1435
1436 - `type TextSimilarityGrader struct{…}`
1437
1438 A TextSimilarityGrader object which grades text based on similarity metrics.
1439
1440 - `type PythonGrader struct{…}`
1441
1442 A PythonGrader object that runs a python script on the input.
1443
1444 - `type ScoreModelGrader struct{…}`
1445
1446 A ScoreModelGrader object that uses a model to assign a score to the input.
1447
1448 - `type LabelModelGrader struct{…}`
1449
1450 A LabelModelGrader object which uses a model to assign labels to each item
1451 in the evaluation.
1452
1453 - `Input []LabelModelGraderInput`
1454
1455 - `Content LabelModelGraderInputContentUnion`
1456
1457 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.
1458
1459 - `string`
1460
1461 - `type ResponseInputText struct{…}`
1462
1463 A text input to the model.
1464
1465 - `type LabelModelGraderInputContentOutputText struct{…}`
1466
1467 A text output from the model.
1468
1469 - `Text string`
1470
1471 The text output from the model.
1472
1473 - `Type OutputText`
1474
1475 The type of the output text. Always `output_text`.
1476
1477 - `const OutputTextOutputText OutputText = "output_text"`
1478
1479 - `type LabelModelGraderInputContentInputImage struct{…}`
1480
1481 An image input block used within EvalItem content arrays.
1482
1483 - `ImageURL string`
1484
1485 The URL of the image input.
1486
1487 - `Type InputImage`
1488
1489 The type of the image input. Always `input_image`.
1490
1491 - `const InputImageInputImage InputImage = "input_image"`
1492
1493 - `Detail string`
1494
1495 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
1496
1497 - `type ResponseInputAudio struct{…}`
1498
1499 An audio input to the model.
1500
1501 - `type GraderInputs []GraderInputUnion`
1502
1503 A list of inputs, each of which may be either an input text, output text, input
1504 image, or input audio object.
1505
1506 - `Role string`
1507
1508 The role of the message input. One of `user`, `assistant`, `system`, or
1509 `developer`.
1510
1511 - `const LabelModelGraderInputRoleUser LabelModelGraderInputRole = "user"`
1512
1513 - `const LabelModelGraderInputRoleAssistant LabelModelGraderInputRole = "assistant"`
1514
1515 - `const LabelModelGraderInputRoleSystem LabelModelGraderInputRole = "system"`
1516
1517 - `const LabelModelGraderInputRoleDeveloper LabelModelGraderInputRole = "developer"`
1518
1519 - `Type string`
1520
1521 The type of the message input. Always `message`.
1522
1523 - `const LabelModelGraderInputTypeMessage LabelModelGraderInputType = "message"`
1524
1525 - `Labels []string`
1526
1527 The labels to assign to each item in the evaluation.
1528
1529 - `Model string`
1530
1531 The model to use for the evaluation. Must support structured outputs.
1532
1533 - `Name string`
1534
1535 The name of the grader.
1536
1537 - `PassingLabels []string`
1538
1539 The labels that indicate a passing result. Must be a subset of labels.
1540
1541 - `Type LabelModel`
1542
1543 The object type, which is always `label_model`.
1544
1545 - `const LabelModelLabelModel LabelModel = "label_model"`
1546
1547 - `Name string`
1548
1549 The name of the grader.
1550
1551 - `Type Multi`
1552
1553 The object type, which is always `multi`.
1554
1555 - `const MultiMulti Multi = "multi"`
1556
1557### Example
1558
1559```go
1560package main
1561
1562import (
1563 "context"
1564 "fmt"
1565
1566 "github.com/openai/openai-go"
1567 "github.com/openai/openai-go/option"
1568)
1569
1570func main() {
1571 client := openai.NewClient(
1572 option.WithAPIKey("My API Key"),
1573 )
1574 response, err := client.FineTuning.Alpha.Graders.Validate(context.TODO(), openai.FineTuningAlphaGraderValidateParams{
1575 Grader: openai.FineTuningAlphaGraderValidateParamsGraderUnion{
1576 OfStringCheckGrader: &openai.StringCheckGraderParam{
1577 Input: "input",
1578 Name: "name",
1579 Operation: openai.StringCheckGraderOperationEq,
1580 Reference: "reference",
1581 },
1582 },
1583 })
1584 if err != nil {
1585 panic(err.Error())
1586 }
1587 fmt.Printf("%+v\n", response.Grader)
1588}
1589```
1590
1591#### Response
1592
1593```json
1594{
1595 "grader": {
1596 "input": "input",
1597 "name": "name",
1598 "operation": "eq",
1599 "reference": "reference",
1600 "type": "string_check"
1601 }
1602}
1603```