Skip to content
Hayonix

API reference · Schema

EvaluationList

  • The items

    Show 25 fields of data
    • idstringrequired

      Unique ID (`evl_…`)

      ID with prefix evl_

    • objectstringrequired

      Always `evaluation`: the type of this object

      One ofevaluation

    • project_idstringrequired

      Project it belongs to

      ID with prefix prj_

    • namestring | nullrequired

      Its name

    • modestringrequired

      `label`: one label per row; `grade`: a score per row; `pairwise`: the better of two candidates per row

      One oflabelgradepairwise

    • statusstringrequired

      validating → running → finalizing → completed | stopped | failed; cancelling → cancelled. `stopped`: ended early (credit, time limit, revoked key) with every finished row kept.

      One ofvalidatingrunningfinalizingcompletedstoppedfailedcancellingcancelled

    • stop_reasonstring | nullrequired

      Why the evaluation ended before every row was finished: cancelled, expired, insufficient_credit, spend_limit_reached, organization_suspended, api_key_revoked, billing_<reason>; null otherwise

    • dataset_file_idstringrequired

      ID of the dataset file

      ID with prefix file_

    • result_file_idstring | nullrequired

      JSONL, one line per row (purpose `evaluation_results`), once ended

    • judgeobjectrequired

      The model that gives each verdict, and its settings

      Show 4 fields of judge
      • modelstringrequired

        Model ID (`publisher/name`)

      • instructionsstringrequired

        What the judge assesses, a template over the row

      • max_tokensintegerrequired

        Most tokens per judge answer

      • temperaturenumberrequired

        The judge's sampling temperature

    • targetEvaluationCandidaterequired
      Show 6 fields of target
      • columnstring | nullrequired

        The dataset column whose text is judged; null when a model generates it

      • modelstring | nullrequired

        The model that generates the text; null when the text comes from a column

      • promptstring | nullrequired

        With `model`: the user message template; null otherwise

      • system_promptstring | nullrequired

        With `model`: the system message template; null when none was given

      • max_tokensinteger | nullrequired

        With `model`: most tokens generated per row; null otherwise

      • temperaturenumber | nullrequired

        With `model`: sampling temperature; null otherwise

    • pairwiseobject | nullrequired

      Mode `pairwise`: the two candidates; null in other modes

      Show 2 fields of pairwise
      • Show 6 fields of a
        • columnstring | nullrequired

          The dataset column whose text is judged; null when a model generates it

        • modelstring | nullrequired

          The model that generates the text; null when the text comes from a column

        • promptstring | nullrequired

          With `model`: the user message template; null otherwise

        • system_promptstring | nullrequired

          With `model`: the system message template; null when none was given

        • max_tokensinteger | nullrequired

          With `model`: most tokens generated per row; null otherwise

        • temperaturenumber | nullrequired

          With `model`: sampling temperature; null otherwise

      • Show 6 fields of b
        • columnstring | nullrequired

          The dataset column whose text is judged; null when a model generates it

        • modelstring | nullrequired

          The model that generates the text; null when the text comes from a column

        • promptstring | nullrequired

          With `model`: the user message template; null otherwise

        • system_promptstring | nullrequired

          With `model`: the system message template; null when none was given

        • max_tokensinteger | nullrequired

          With `model`: most tokens generated per row; null otherwise

        • temperaturenumber | nullrequired

          With `model`: sampling temperature; null otherwise

    • labelobject | nullrequired

      Mode `label`: its settings; null in other modes

      Show 2 fields of label
      • choicesarray of stringrequired

        The labels the judge chooses from

      • passingarray of stringrequired

        Labels that count as passing; empty when no pass rate is computed

    • gradeobject | nullrequired

      Mode `grade`: its settings; null in other modes

      Show 3 fields of grade
      • minnumberrequired

        Lowest allowed score (inclusive)

      • maxnumberrequired

        Highest allowed score (inclusive)

      • passing_atnumber | nullrequired

        Time passing (ISO 8601, UTC)

    • progressobjectrequired

      How far the run has got

      Show 6 fields of progress
      • rows_totalintegerrequired

        Rows in the dataset; 0 until validation has finished

      • rows_judgedintegerrequired

        Rows that received a verdict

      • rows_erroredintegerrequired

        Rows that could not be judged

      • rows_skippedintegerrequired

        Rows never finished because the run ended early

      • calls_completedintegerrequired

        Model calls finished so far, retries included

      • calls_plannedintegerrequired

        Rows × calls per row, before retries

    • resultsEvaluationResultsrequired

      Aggregate results so far (final once ended)

      Show 6 fields of results
      • rowsobjectrequired

        Rows by status

        Show 5 fields of rows
        • totalintegerrequired

          Rows in the dataset

        • judgedintegerrequired

          Rows that received a verdict

        • erroredintegerrequired

          Rows that could not be judged (a failed call, a missing column, no usable judge answer)

        • skippedintegerrequired

          Never finished: the run ended early

        • openintegerrequired

          Rows still being worked on

      • passedinteger | nullrequired

        Judged rows that pass; null when the evaluation defines no passing labels or score, and in mode `pairwise`

      • pass_ratenumber | nullrequired

        Passed ÷ judged, 0–1

      • labelsarray of object | nullrequired

        Mode `label`: every choice with its count, zeros included; null in other modes

        Show 2 fields of labels
        • labelstringrequired

          One of `label.choices`

        • countintegerrequired

          Judged rows given this label

      • scoresobject | nullrequired

        Mode `grade`: statistics of the scores; null in other modes

        Show 5 fields of scores
        • meannumber | nullrequired

          Mean score; null before any row is judged

        • std_devnumber | nullrequired

          Population standard deviation of the scores; null before any row is judged

        • minnumber | nullrequired

          Lowest score given; null before any row is judged

        • maxnumber | nullrequired

          Highest score given; null before any row is judged

        • histogramarray of objectrequired

          Scores over `grade.min`–`grade.max`: one bucket per whole number when there are at most 20, else 20 equal-width buckets

      • pairwiseobject | nullrequired

        Mode `pairwise`: how the two candidates compared; null in other modes

        Show 5 fields of pairwise
        • a_winsintegerrequired

          Pairs where both passes chose A

        • b_winsintegerrequired

          Pairs where both passes chose B

        • tiesintegerrequired

          Pairs where both passes called a tie

        • inconsistentintegerrequired

          Pairs whose two judging passes (A first, B first) disagreed

        • a_win_ratenumber | nullrequired

          A wins ÷ (A wins + B wins + ties), 0–1, over consistent pairs only; null before any consistent pair

    • usageobjectrequired

      Tokens of every model call completed so far

      Show 3 fields of usage
      • input_tokensintegerrequired

        Prompt tokens over all completed calls

      • cached_input_tokensintegerrequired

        Of `input_tokens`, those served from the prompt cache

      • output_tokensintegerrequired

        Generated tokens over all completed calls

    • coststringrequired

      Charged so far (every call, at batch prices)

      pattern ^-?[0-9]+$

    • problemsarray of objectrequired

      Why the evaluation failed

      Show 4 fields of problems
      • codestringrequired

        Machine-readable code

      • messagestringrequired

        One sentence for people

      • paramstring | nullrequired

        The request field at fault; null when not about one field

      • lineinteger | nullrequired

        1-based line of the dataset file; null when not about one line

    • created_atstring (date-time)required

      When it was created (ISO 8601, UTC)

    • running_atstring (date-time) | nullrequired

      Time running (ISO 8601, UTC)

    • finalizing_atstring (date-time) | nullrequired

      Time finalizing (ISO 8601, UTC)

    • cancelling_atstring (date-time) | nullrequired

      Time cancelling (ISO 8601, UTC)

    • ended_atstring (date-time) | nullrequired

      When it ended (null until then) (ISO 8601, UTC)

    • expires_atstring (date-time)required

      When a run still going stops

  • next_cursorstring | nullrequired

    Pass as `cursor` to get the next page; null when there is no more

Example
{
  "data": [
    {
      "id": "evl_01jc5zq4m8k3v6x9t2w7r0b1e4",
      "object": "evaluation",
      "project_id": "prj_01jc5zq4m8k3v6x9t2w7r0b1e4",
      "name": "string",
      "mode": "label",
      "status": "validating",
      "stop_reason": "string",
      "dataset_file_id": "file_01jc5zq4m8k3v6x9t2w7r0b1e4",
      "result_file_id": "string",
      "judge": {
        "model": "string",
        "instructions": "string",
        "max_tokens": 1,
        "temperature": 1.5
      },
      "target": {
        "column": "string",
        "model": "string",
        "prompt": "string",
        "system_prompt": "string",
        "max_tokens": 1,
        "temperature": 1.5
      },
      "pairwise": {
        "a": {
          "column": "string",
          "model": "string",
          "prompt": "string",
          "system_prompt": "string",
          "max_tokens": 1,
          "temperature": 1.5
        },
        "b": {
          "column": "string",
          "model": "string",
          "prompt": "string",
          "system_prompt": "string",
          "max_tokens": 1,
          "temperature": 1.5
        }
      },
      "label": {
        "choices": [
          "string"
        ],
        "passing": [
          "string"
        ]
      },
      "grade": {
        "min": 1.5,
        "max": 1.5,
        "passing_at": 1.5
      },
      "progress": {
        "rows_total": 1,
        "rows_judged": 1,
        "rows_errored": 1,
        "rows_skipped": 1,
        "calls_completed": 1,
        "calls_planned": 1
      },
      "results": {
        "rows": {
          "total": 1,
          "judged": 1,
          "errored": 1,
          "skipped": 1,
          "open": 1
        },
        "passed": 1,
        "pass_rate": 1.5,
        "labels": [
          {
            "label": "string",
            "count": 1
          }
        ],
        "scores": {
          "mean": 1.5,
          "std_dev": 1.5,
          "min": 1.5,
          "max": 1.5,
          "histogram": [
            {
              "from": null,
              "to": null,
              "count": null
            }
          ]
        },
        "pairwise": {
          "a_wins": 1,
          "b_wins": 1,
          "ties": 1,
          "inconsistent": 1,
          "a_win_rate": 1.5
        }
      },
      "usage": {
        "input_tokens": 1,
        "cached_input_tokens": 1,
        "output_tokens": 1
      },
      "cost": "1500000000",
      "problems": [
        {
          "code": "string",
          "message": "string",
          "param": "string",
          "line": 1
        }
      ],
      "created_at": "2026-01-01T00:00:00.000Z",
      "running_at": "2026-01-01T00:00:00.000Z",
      "finalizing_at": "2026-01-01T00:00:00.000Z",
      "cancelling_at": "2026-01-01T00:00:00.000Z",
      "ended_at": "2026-01-01T00:00:00.000Z",
      "expires_at": "2026-01-01T00:00:00.000Z"
    }
  ],
  "next_cursor": "string"
}

Used by