Skip to content
Hayonix

API reference · Schema

EvaluationRowList

  • The items

    Show 13 fields of data
    • idstringrequired

      Unique ID (`evr_…`)

      ID with prefix evr_

    • objectstringrequired

      Always `evaluation_row`: the type of this object

      One ofevaluation_row

    • rowintegerrequired

      1-based data row

    • lineintegerrequired

      1-based line of the dataset file

    • statusstringrequired

      `open`: still being worked on; `judged`: has a verdict; `errored`: could not be judged (see `error`); `skipped`: never finished because the run ended early

      One ofopenjudgederroredskipped

    • inputsobjectrequired

      The columns the run used (each cut to 2000 characters; the results file has them whole)

    • generatedobjectrequired

      Generated texts by candidate (`target`, `a`, `b`), cut the same way

    • verdictEvaluationVerdictrequired
      Show 4 fields of verdict
      • labelstring

        Mode `label`: the chosen label

      • scorenumber

        Mode `grade`: the score given

      • winnerstring

        Mode `pairwise`: `a`, `b` or `tie` when both judging passes agree; `inconsistent` when they disagree

        One ofabtieinconsistent

      • passesobject

        Pairwise: the winner of each judging pass

        Show 2 fields of passes
        • a_firststringrequired

          The winner when A was shown to the judge first

          One ofabtie

        • b_firststringrequired

          The winner when B was shown to the judge first

          One ofabtie

    • passedboolean | nullrequired

      Whether the verdict passes; null until judged, when the evaluation defines no passing labels or score, and in mode `pairwise`

    • reasoningstring | nullrequired

      The judge's explanation (pairwise: one per judging pass); null until judged

    • errorobject | nullrequired

      Why the row was errored or skipped; null otherwise

      Show 2 fields of error
      • codestringrequired

        Machine-readable code

      • messagestringrequired

        One sentence for people

    • usageobjectrequired

      This row's model calls and tokens; all 0 until the row is finished

      Show 3 fields of usage
      • callsintegerrequired

        Model calls made for this row, every try included

      • input_tokensintegerrequired

        Prompt tokens over those calls

      • output_tokensintegerrequired

        Generated tokens over those calls

    • coststringrequired

      Integer nano-USD as a decimal string

      pattern ^-?[0-9]+$

  • next_cursorstring | nullrequired

    Pass as `cursor` to get the next page; null when there is no more

Example
{
  "data": [
    {
      "id": "evr_01jc5zq4m8k3v6x9t2w7r0b1e4",
      "object": "evaluation_row",
      "row": 1,
      "line": 1,
      "status": "open",
      "inputs": {},
      "generated": {},
      "verdict": {
        "label": "string",
        "score": 1.5,
        "winner": "a",
        "passes": {
          "a_first": "a",
          "b_first": "a"
        }
      },
      "passed": true,
      "reasoning": "string",
      "error": {
        "code": "string",
        "message": "string"
      },
      "usage": {
        "calls": 1,
        "input_tokens": 1,
        "output_tokens": 1
      },
      "cost": "1500000000"
    }
  ],
  "next_cursor": "string"
}

Used by