ruby/resources/evals/methods/retrieve/index.md 2026-07-10 23:02 UTC to 2026-07-12 06:58 UTC

Get an eval

evals.retrieve(eval_id) -> EvalRetrieveResponse

get /evals/{eval_id}

Get an evaluation by ID.

Parameters

• eval_id: String

Returns

• class EvalRetrieveResponse

  An Eval object with a data source config and testing criteria. An Eval represents a task to be done for your LLM integration. Like:

  • Improve the quality of my chatbot

  • See how well my chatbot handles customer support

  • Check if o4-mini is better at my usecase than gpt-4o

  • id: String

    Unique identifier for the evaluation.

  • created_at: Integer

    The Unix timestamp (in seconds) for when the eval was created.

  • data_source_config: EvalCustomDataSourceConfig | Logs{ schema, type, metadata} | EvalStoredCompletionsDataSourceConfig

    Configuration of data sources used in runs of the evaluation.

    • class EvalCustomDataSourceConfig

      A CustomDataSourceConfig which specifies the schema of your item and optionally sample namespaces. The response schema defines the shape of the data that will be:

      • Used to define your testing criteria and

      • What data is required when creating a run

      • schema: Hash[Symbol, untyped]

        The json schema for the run data source items. Learn how to build JSON schemas here.

      • type: :custom

        The type of data source. Always custom.

        • :custom

    • class Logs

      A LogsDataSourceConfig which specifies the metadata property of your logs query. This is usually metadata like usecase=chatbot or prompt-version=v2, etc. The schema returned by this data source config is used to defined what variables are available in your evals. item and sample are both defined when using this data source config.

      • schema: Hash[Symbol, untyped]

        The json schema for the run data source items. Learn how to build JSON schemas here.

      • type: :logs

        The type of data source. Always logs.

        • :logs

      • metadata: Metadata

        Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard.

        Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.

    • class EvalStoredCompletionsDataSourceConfig

      Deprecated in favor of LogsDataSourceConfig.

      • schema: Hash[Symbol, untyped]

        The json schema for the run data source items. Learn how to build JSON schemas here.

      • type: :stored_completions

        The type of data source. Always stored_completions.

        • :stored_completions

      • metadata: Metadata

        Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard.

        Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.

  • metadata: Metadata

    Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard.

    Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.

  • name: String

    The name of the evaluation.

  • object: :eval

    The object type.

    • :eval

  • testing_criteria: Array[LabelModelGrader | StringCheckGrader | TextSimilarityGrader & { pass_threshold} | 2 more]

    A list of testing criteria.

    • class LabelModelGrader

      A LabelModelGrader object which uses a model to assign labels to each item in the evaluation.

      • input: Array[Input{ content, role, type}]

        • content: String | ResponseInputText | OutputText{ text, type} | 3 more

          Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

          • String = String

            A text input to the model.

          • class ResponseInputText

            A text input to the model.

            • text: String

              The text input to the model.

            • type: :input_text

              The type of the input item. Always input_text.

              • :input_text

            • prompt_cache_breakpoint: PromptCacheBreakpoint{ mode}

              Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's prompt_cache_options.ttl; the boundary is not rounded to a token block.

              • mode: :explicit

                The breakpoint mode. Always explicit.

                • :explicit

          • class OutputText

            A text output from the model.

            • text: String

              The text output from the model.

            • type: :output_text

              The type of the output text. Always output_text.

              • :output_text

          • class InputImage

            An image input block used within EvalItem content arrays.

            • image_url: String

              The URL of the image input.

            • type: :input_image

              The type of the image input. Always input_image.

              • :input_image

            • detail: String

              The detail level of the image to be sent to the model. One of high, low, or auto. Defaults to auto.

          • class ResponseInputAudio

            An audio input to the model.

            • input_audio: InputAudio{ data, format_}

              • data: String

                Base64-encoded audio data.

              • format_: :mp3 | :wav

                The format of the audio data. Currently supported formats are mp3 and wav.

                • :mp3

                • :wav

            • type: :input_audio

              The type of the input item. Always input_audio.

              • :input_audio

          • GraderInputs = Array[GraderInputItem]

            A list of inputs, each of which may be either an input text, output text, input image, or input audio object.

            • String = String

              A text input to the model.

            • class ResponseInputText

              A text input to the model.

            • class OutputText

              A text output from the model.

              • text: String

                The text output from the model.

              • type: :output_text

                The type of the output text. Always output_text.

                • :output_text

            • class InputImage

              An image input block used within EvalItem content arrays.

              • image_url: String

                The URL of the image input.

              • type: :input_image

                The type of the image input. Always input_image.

                • :input_image

              • detail: String

                The detail level of the image to be sent to the model. One of high, low, or auto. Defaults to auto.

            • class ResponseInputAudio

              An audio input to the model.

        • role: :user | :assistant | :system | :developer

          The role of the message input. One of user, assistant, system, or developer.

          • :user

          • :assistant

          • :system

          • :developer

        • type: :message

          The type of the message input. Always message.

          • :message

      • labels: Array[String]

        The labels to assign to each item in the evaluation.

      • model: String

        The model to use for the evaluation. Must support structured outputs.

      • name: String

        The name of the grader.

      • passing_labels: Array[String]

        The labels that indicate a passing result. Must be a subset of labels.

      • type: :label_model

        The object type, which is always label_model.

        • :label_model

    • class StringCheckGrader

      A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.

      • input: String

        The input text. This may include template strings.

      • name: String

        The name of the grader.

      • operation: :eq | :ne | :like | :ilike

        The string check operation to perform. One of eq, ne, like, or ilike.

        • :eq

        • :ne

        • :like

        • :ilike

      • reference: String

        The reference text. This may include template strings.

      • type: :string_check

        The object type, which is always string_check.

        • :string_check

    • class EvalGraderTextSimilarity

      A TextSimilarityGrader object which grades text based on similarity metrics.

      • pass_threshold: Float

        The threshold for the score.

    • class EvalGraderPython

      A PythonGrader object that runs a python script on the input.

      • pass_threshold: Float

        The threshold for the score.

    • class EvalGraderScoreModel

      A ScoreModelGrader object that uses a model to assign a score to the input.

      • pass_threshold: Float

        The threshold for the score.

Example

require "openai"

openai = OpenAI::Client.new(api_key: "My API Key")

eval_ = openai.evals.retrieve("eval_id")

puts(eval_)

Response

{
  "id": "id",
  "created_at": 0,
  "data_source_config": {
    "schema": {
      "foo": "bar"
    },
    "type": "custom"
  },
  "metadata": {
    "foo": "string"
  },
  "name": "Chatbot effectiveness Evaluation",
  "object": "eval",
  "testing_criteria": [
    {
      "input": [
        {
          "content": "string",
          "role": "user",
          "type": "message"
        }
      ],
      "labels": [
        "string"
      ],
      "model": "model",
      "name": "name",
      "passing_labels": [
        "string"
      ],
      "type": "label_model"
    }
  ]
}