java/resources/evals/methods/retrieve/index.md +0 −424 deleted
File Deleted View Diff
1## Get an eval
2
3`EvalRetrieveResponse evals().retrieve(EvalRetrieveParamsparams = EvalRetrieveParams.none(), RequestOptionsrequestOptions = RequestOptions.none())`
4
5**get** `/evals/{eval_id}`
6
7Get an evaluation by ID.
8
9### Parameters
10
11- `EvalRetrieveParams params`
12
13 - `Optional<String> evalId`
14
15### Returns
16
17- `class EvalRetrieveResponse:`
18
19 An Eval object with a data source config and testing criteria.
20 An Eval represents a task to be done for your LLM integration.
21 Like:
22
23 - Improve the quality of my chatbot
24 - See how well my chatbot handles customer support
25 - Check if o4-mini is better at my usecase than gpt-4o
26
27 - `String id`
28
29 Unique identifier for the evaluation.
30
31 - `long createdAt`
32
33 The Unix timestamp (in seconds) for when the eval was created.
34
35 - `DataSourceConfig dataSourceConfig`
36
37 Configuration of data sources used in runs of the evaluation.
38
39 - `class EvalCustomDataSourceConfig:`
40
41 A CustomDataSourceConfig which specifies the schema of your `item` and optionally `sample` namespaces.
42 The response schema defines the shape of the data that will be:
43
44 - Used to define your testing criteria and
45 - What data is required when creating a run
46
47 - `Schema schema`
48
49 The json schema for the run data source items.
50 Learn how to build JSON schemas [here](https://json-schema.org/).
51
52 - `JsonValue; type "custom"constant`
53
54 The type of data source. Always `custom`.
55
56 - `CUSTOM("custom")`
57
58 - `class Logs:`
59
60 A LogsDataSourceConfig which specifies the metadata property of your logs query.
61 This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc.
62 The schema returned by this data source config is used to defined what variables are available in your evals.
63 `item` and `sample` are both defined when using this data source config.
64
65 - `Schema schema`
66
67 The json schema for the run data source items.
68 Learn how to build JSON schemas [here](https://json-schema.org/).
69
70 - `JsonValue; type "logs"constant`
71
72 The type of data source. Always `logs`.
73
74 - `LOGS("logs")`
75
76 - `Optional<Metadata> metadata`
77
78 Set of 16 key-value pairs that can be attached to an object. This can be
79 useful for storing additional information about the object in a structured
80 format, and querying for objects via API or the dashboard.
81
82 Keys are strings with a maximum length of 64 characters. Values are strings
83 with a maximum length of 512 characters.
84
85 - `class EvalStoredCompletionsDataSourceConfig:`
86
87 Deprecated in favor of LogsDataSourceConfig.
88
89 - `Schema schema`
90
91 The json schema for the run data source items.
92 Learn how to build JSON schemas [here](https://json-schema.org/).
93
94 - `JsonValue; type "stored_completions"constant`
95
96 The type of data source. Always `stored_completions`.
97
98 - `STORED_COMPLETIONS("stored_completions")`
99
100 - `Optional<Metadata> metadata`
101
102 Set of 16 key-value pairs that can be attached to an object. This can be
103 useful for storing additional information about the object in a structured
104 format, and querying for objects via API or the dashboard.
105
106 Keys are strings with a maximum length of 64 characters. Values are strings
107 with a maximum length of 512 characters.
108
109 - `Optional<Metadata> metadata`
110
111 Set of 16 key-value pairs that can be attached to an object. This can be
112 useful for storing additional information about the object in a structured
113 format, and querying for objects via API or the dashboard.
114
115 Keys are strings with a maximum length of 64 characters. Values are strings
116 with a maximum length of 512 characters.
117
118 - `String name`
119
120 The name of the evaluation.
121
122 - `JsonValue; object_ "eval"constant`
123
124 The object type.
125
126 - `EVAL("eval")`
127
128 - `List<TestingCriterion> testingCriteria`
129
130 A list of testing criteria.
131
132 - `class LabelModelGrader:`
133
134 A LabelModelGrader object which uses a model to assign labels to each item
135 in the evaluation.
136
137 - `List<Input> input`
138
139 - `Content content`
140
141 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.
142
143 - `String`
144
145 - `class ResponseInputText:`
146
147 A text input to the model.
148
149 - `String text`
150
151 The text input to the model.
152
153 - `JsonValue; type "input_text"constant`
154
155 The type of the input item. Always `input_text`.
156
157 - `INPUT_TEXT("input_text")`
158
159 - `Optional<PromptCacheBreakpoint> promptCacheBreakpoint`
160
161 Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.
162
163 - `JsonValue; mode "explicit"constant`
164
165 The breakpoint mode. Always `explicit`.
166
167 - `EXPLICIT("explicit")`
168
169 - `class OutputText:`
170
171 A text output from the model.
172
173 - `String text`
174
175 The text output from the model.
176
177 - `JsonValue; type "output_text"constant`
178
179 The type of the output text. Always `output_text`.
180
181 - `OUTPUT_TEXT("output_text")`
182
183 - `class InputImage:`
184
185 An image input block used within EvalItem content arrays.
186
187 - `String imageUrl`
188
189 The URL of the image input.
190
191 - `JsonValue; type "input_image"constant`
192
193 The type of the image input. Always `input_image`.
194
195 - `INPUT_IMAGE("input_image")`
196
197 - `Optional<String> detail`
198
199 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
200
201 - `class ResponseInputAudio:`
202
203 An audio input to the model.
204
205 - `InputAudio inputAudio`
206
207 - `String data`
208
209 Base64-encoded audio data.
210
211 - `Format format`
212
213 The format of the audio data. Currently supported formats are `mp3` and
214 `wav`.
215
216 - `MP3("mp3")`
217
218 - `WAV("wav")`
219
220 - `JsonValue; type "input_audio"constant`
221
222 The type of the input item. Always `input_audio`.
223
224 - `INPUT_AUDIO("input_audio")`
225
226 - `List<EvalContentItem>`
227
228 - `String`
229
230 - `class ResponseInputText:`
231
232 A text input to the model.
233
234 - `OutputText`
235
236 - `String text`
237
238 The text output from the model.
239
240 - `JsonValue; type "output_text"constant`
241
242 The type of the output text. Always `output_text`.
243
244 - `OUTPUT_TEXT("output_text")`
245
246 - `InputImage`
247
248 - `String imageUrl`
249
250 The URL of the image input.
251
252 - `JsonValue; type "input_image"constant`
253
254 The type of the image input. Always `input_image`.
255
256 - `INPUT_IMAGE("input_image")`
257
258 - `Optional<String> detail`
259
260 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
261
262 - `class ResponseInputAudio:`
263
264 An audio input to the model.
265
266 - `Role role`
267
268 The role of the message input. One of `user`, `assistant`, `system`, or
269 `developer`.
270
271 - `USER("user")`
272
273 - `ASSISTANT("assistant")`
274
275 - `SYSTEM("system")`
276
277 - `DEVELOPER("developer")`
278
279 - `Optional<Type> type`
280
281 The type of the message input. Always `message`.
282
283 - `MESSAGE("message")`
284
285 - `List<String> labels`
286
287 The labels to assign to each item in the evaluation.
288
289 - `String model`
290
291 The model to use for the evaluation. Must support structured outputs.
292
293 - `String name`
294
295 The name of the grader.
296
297 - `List<String> passingLabels`
298
299 The labels that indicate a passing result. Must be a subset of labels.
300
301 - `JsonValue; type "label_model"constant`
302
303 The object type, which is always `label_model`.
304
305 - `LABEL_MODEL("label_model")`
306
307 - `class StringCheckGrader:`
308
309 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
310
311 - `String input`
312
313 The input text. This may include template strings.
314
315 - `String name`
316
317 The name of the grader.
318
319 - `Operation operation`
320
321 The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
322
323 - `EQ("eq")`
324
325 - `NE("ne")`
326
327 - `LIKE("like")`
328
329 - `ILIKE("ilike")`
330
331 - `String reference`
332
333 The reference text. This may include template strings.
334
335 - `JsonValue; type "string_check"constant`
336
337 The object type, which is always `string_check`.
338
339 - `STRING_CHECK("string_check")`
340
341 - `class EvalGraderTextSimilarity:`
342
343 A TextSimilarityGrader object which grades text based on similarity metrics.
344
345 - `double passThreshold`
346
347 The threshold for the score.
348
349 - `class EvalGraderPython:`
350
351 A PythonGrader object that runs a python script on the input.
352
353 - `Optional<Double> passThreshold`
354
355 The threshold for the score.
356
357 - `class EvalGraderScoreModel:`
358
359 A ScoreModelGrader object that uses a model to assign a score to the input.
360
361 - `Optional<Double> passThreshold`
362
363 The threshold for the score.
364
365### Example
366
367```java
368package com.openai.example;
369
370import com.openai.client.OpenAIClient;
371import com.openai.client.okhttp.OpenAIOkHttpClient;
372import com.openai.models.evals.EvalRetrieveParams;
373import com.openai.models.evals.EvalRetrieveResponse;
374
375public final class Main {
376 private Main() {}
377
378 public static void main(String[] args) {
379 OpenAIClient client = OpenAIOkHttpClient.fromEnv();
380
381 EvalRetrieveResponse eval = client.evals().retrieve("eval_id");
382 }
383}
384```
385
386#### Response
387
388```json
389{
390 "id": "id",
391 "created_at": 0,
392 "data_source_config": {
393 "schema": {
394 "foo": "bar"
395 },
396 "type": "custom"
397 },
398 "metadata": {
399 "foo": "string"
400 },
401 "name": "Chatbot effectiveness Evaluation",
402 "object": "eval",
403 "testing_criteria": [
404 {
405 "input": [
406 {
407 "content": "string",
408 "role": "user",
409 "type": "message"
410 }
411 ],
412 "labels": [
413 "string"
414 ],
415 "model": "model",
416 "name": "name",
417 "passing_labels": [
418 "string"
419 ],
420 "type": "label_model"
421 }
422 ]
423}
424```