ruby/resources/evals/methods/update/index.md +0 −437 deleted
File Deleted View Diff
1## Update an eval
2
3`evals.update(eval_id, **kwargs) -> EvalUpdateResponse`
4
5**post** `/evals/{eval_id}`
6
7Update certain properties of an evaluation.
8
9### Parameters
10
11- `eval_id: String`
12
13- `metadata: Metadata`
14
15 Set of 16 key-value pairs that can be attached to an object. This can be
16 useful for storing additional information about the object in a structured
17 format, and querying for objects via API or the dashboard.
18
19 Keys are strings with a maximum length of 64 characters. Values are strings
20 with a maximum length of 512 characters.
21
22- `name: String`
23
24 Rename the evaluation.
25
26### Returns
27
28- `class EvalUpdateResponse`
29
30 An Eval object with a data source config and testing criteria.
31 An Eval represents a task to be done for your LLM integration.
32 Like:
33
34 - Improve the quality of my chatbot
35 - See how well my chatbot handles customer support
36 - Check if o4-mini is better at my usecase than gpt-4o
37
38 - `id: String`
39
40 Unique identifier for the evaluation.
41
42 - `created_at: Integer`
43
44 The Unix timestamp (in seconds) for when the eval was created.
45
46 - `data_source_config: EvalCustomDataSourceConfig | Logs{ schema, type, metadata} | EvalStoredCompletionsDataSourceConfig`
47
48 Configuration of data sources used in runs of the evaluation.
49
50 - `class EvalCustomDataSourceConfig`
51
52 A CustomDataSourceConfig which specifies the schema of your `item` and optionally `sample` namespaces.
53 The response schema defines the shape of the data that will be:
54
55 - Used to define your testing criteria and
56 - What data is required when creating a run
57
58 - `schema: Hash[Symbol, untyped]`
59
60 The json schema for the run data source items.
61 Learn how to build JSON schemas [here](https://json-schema.org/).
62
63 - `type: :custom`
64
65 The type of data source. Always `custom`.
66
67 - `:custom`
68
69 - `class Logs`
70
71 A LogsDataSourceConfig which specifies the metadata property of your logs query.
72 This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc.
73 The schema returned by this data source config is used to defined what variables are available in your evals.
74 `item` and `sample` are both defined when using this data source config.
75
76 - `schema: Hash[Symbol, untyped]`
77
78 The json schema for the run data source items.
79 Learn how to build JSON schemas [here](https://json-schema.org/).
80
81 - `type: :logs`
82
83 The type of data source. Always `logs`.
84
85 - `:logs`
86
87 - `metadata: Metadata`
88
89 Set of 16 key-value pairs that can be attached to an object. This can be
90 useful for storing additional information about the object in a structured
91 format, and querying for objects via API or the dashboard.
92
93 Keys are strings with a maximum length of 64 characters. Values are strings
94 with a maximum length of 512 characters.
95
96 - `class EvalStoredCompletionsDataSourceConfig`
97
98 Deprecated in favor of LogsDataSourceConfig.
99
100 - `schema: Hash[Symbol, untyped]`
101
102 The json schema for the run data source items.
103 Learn how to build JSON schemas [here](https://json-schema.org/).
104
105 - `type: :stored_completions`
106
107 The type of data source. Always `stored_completions`.
108
109 - `:stored_completions`
110
111 - `metadata: Metadata`
112
113 Set of 16 key-value pairs that can be attached to an object. This can be
114 useful for storing additional information about the object in a structured
115 format, and querying for objects via API or the dashboard.
116
117 Keys are strings with a maximum length of 64 characters. Values are strings
118 with a maximum length of 512 characters.
119
120 - `metadata: Metadata`
121
122 Set of 16 key-value pairs that can be attached to an object. This can be
123 useful for storing additional information about the object in a structured
124 format, and querying for objects via API or the dashboard.
125
126 Keys are strings with a maximum length of 64 characters. Values are strings
127 with a maximum length of 512 characters.
128
129 - `name: String`
130
131 The name of the evaluation.
132
133 - `object: :eval`
134
135 The object type.
136
137 - `:eval`
138
139 - `testing_criteria: Array[LabelModelGrader | StringCheckGrader | TextSimilarityGrader & { pass_threshold} | 2 more]`
140
141 A list of testing criteria.
142
143 - `class LabelModelGrader`
144
145 A LabelModelGrader object which uses a model to assign labels to each item
146 in the evaluation.
147
148 - `input: Array[Input{ content, role, type}]`
149
150 - `content: String | ResponseInputText | OutputText{ text, type} | 3 more`
151
152 Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.
153
154 - `String = String`
155
156 A text input to the model.
157
158 - `class ResponseInputText`
159
160 A text input to the model.
161
162 - `text: String`
163
164 The text input to the model.
165
166 - `type: :input_text`
167
168 The type of the input item. Always `input_text`.
169
170 - `:input_text`
171
172 - `prompt_cache_breakpoint: PromptCacheBreakpoint{ mode}`
173
174 Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.
175
176 - `mode: :explicit`
177
178 The breakpoint mode. Always `explicit`.
179
180 - `:explicit`
181
182 - `class OutputText`
183
184 A text output from the model.
185
186 - `text: String`
187
188 The text output from the model.
189
190 - `type: :output_text`
191
192 The type of the output text. Always `output_text`.
193
194 - `:output_text`
195
196 - `class InputImage`
197
198 An image input block used within EvalItem content arrays.
199
200 - `image_url: String`
201
202 The URL of the image input.
203
204 - `type: :input_image`
205
206 The type of the image input. Always `input_image`.
207
208 - `:input_image`
209
210 - `detail: String`
211
212 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
213
214 - `class ResponseInputAudio`
215
216 An audio input to the model.
217
218 - `input_audio: InputAudio{ data, format_}`
219
220 - `data: String`
221
222 Base64-encoded audio data.
223
224 - `format_: :mp3 | :wav`
225
226 The format of the audio data. Currently supported formats are `mp3` and
227 `wav`.
228
229 - `:mp3`
230
231 - `:wav`
232
233 - `type: :input_audio`
234
235 The type of the input item. Always `input_audio`.
236
237 - `:input_audio`
238
239 - `GraderInputs = Array[GraderInputItem]`
240
241 A list of inputs, each of which may be either an input text, output text, input
242 image, or input audio object.
243
244 - `String = String`
245
246 A text input to the model.
247
248 - `class ResponseInputText`
249
250 A text input to the model.
251
252 - `class OutputText`
253
254 A text output from the model.
255
256 - `text: String`
257
258 The text output from the model.
259
260 - `type: :output_text`
261
262 The type of the output text. Always `output_text`.
263
264 - `:output_text`
265
266 - `class InputImage`
267
268 An image input block used within EvalItem content arrays.
269
270 - `image_url: String`
271
272 The URL of the image input.
273
274 - `type: :input_image`
275
276 The type of the image input. Always `input_image`.
277
278 - `:input_image`
279
280 - `detail: String`
281
282 The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.
283
284 - `class ResponseInputAudio`
285
286 An audio input to the model.
287
288 - `role: :user | :assistant | :system | :developer`
289
290 The role of the message input. One of `user`, `assistant`, `system`, or
291 `developer`.
292
293 - `:user`
294
295 - `:assistant`
296
297 - `:system`
298
299 - `:developer`
300
301 - `type: :message`
302
303 The type of the message input. Always `message`.
304
305 - `:message`
306
307 - `labels: Array[String]`
308
309 The labels to assign to each item in the evaluation.
310
311 - `model: String`
312
313 The model to use for the evaluation. Must support structured outputs.
314
315 - `name: String`
316
317 The name of the grader.
318
319 - `passing_labels: Array[String]`
320
321 The labels that indicate a passing result. Must be a subset of labels.
322
323 - `type: :label_model`
324
325 The object type, which is always `label_model`.
326
327 - `:label_model`
328
329 - `class StringCheckGrader`
330
331 A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
332
333 - `input: String`
334
335 The input text. This may include template strings.
336
337 - `name: String`
338
339 The name of the grader.
340
341 - `operation: :eq | :ne | :like | :ilike`
342
343 The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
344
345 - `:eq`
346
347 - `:ne`
348
349 - `:like`
350
351 - `:ilike`
352
353 - `reference: String`
354
355 The reference text. This may include template strings.
356
357 - `type: :string_check`
358
359 The object type, which is always `string_check`.
360
361 - `:string_check`
362
363 - `class EvalGraderTextSimilarity`
364
365 A TextSimilarityGrader object which grades text based on similarity metrics.
366
367 - `pass_threshold: Float`
368
369 The threshold for the score.
370
371 - `class EvalGraderPython`
372
373 A PythonGrader object that runs a python script on the input.
374
375 - `pass_threshold: Float`
376
377 The threshold for the score.
378
379 - `class EvalGraderScoreModel`
380
381 A ScoreModelGrader object that uses a model to assign a score to the input.
382
383 - `pass_threshold: Float`
384
385 The threshold for the score.
386
387### Example
388
389```ruby
390require "openai"
391
392openai = OpenAI::Client.new(api_key: "My API Key")
393
394eval_ = openai.evals.update("eval_id")
395
396puts(eval_)
397```
398
399#### Response
400
401```json
402{
403 "id": "id",
404 "created_at": 0,
405 "data_source_config": {
406 "schema": {
407 "foo": "bar"
408 },
409 "type": "custom"
410 },
411 "metadata": {
412 "foo": "string"
413 },
414 "name": "Chatbot effectiveness Evaluation",
415 "object": "eval",
416 "testing_criteria": [
417 {
418 "input": [
419 {
420 "content": "string",
421 "role": "user",
422 "type": "message"
423 }
424 ],
425 "labels": [
426 "string"
427 ],
428 "model": "model",
429 "name": "name",
430 "passing_labels": [
431 "string"
432 ],
433 "type": "label_model"
434 }
435 ]
436}
437```