Skip to content
Hayonix

API reference · Schema

EvaluationRow

  • idstringrequired

    Unique ID (`evr_…`)

    ID with prefix evr_

  • objectstringrequired

    Always `evaluation_row`: the type of this object

    One ofevaluation_row

  • rowintegerrequired

    1-based data row

  • lineintegerrequired

    1-based line of the dataset file

  • statusstringrequired

    `open`: still being worked on; `judged`: has a verdict; `errored`: could not be judged (see `error`); `skipped`: never finished because the run ended early

    One ofopenjudgederroredskipped

  • inputsobjectrequired

    The columns the run used (each cut to 2000 characters; the results file has them whole)

  • generatedobjectrequired

    Generated texts by candidate (`target`, `a`, `b`), cut the same way

  • verdictEvaluationVerdictrequired
    Show 4 fields of verdict
    • labelstring

      Mode `label`: the chosen label

    • scorenumber

      Mode `grade`: the score given

    • winnerstring

      Mode `pairwise`: `a`, `b` or `tie` when both judging passes agree; `inconsistent` when they disagree

      One ofabtieinconsistent

    • passesobject

      Pairwise: the winner of each judging pass

      Show 2 fields of passes
      • a_firststringrequired

        The winner when A was shown to the judge first

        One ofabtie

      • b_firststringrequired

        The winner when B was shown to the judge first

        One ofabtie

  • passedboolean | nullrequired

    Whether the verdict passes; null until judged, when the evaluation defines no passing labels or score, and in mode `pairwise`

  • reasoningstring | nullrequired

    The judge's explanation (pairwise: one per judging pass); null until judged

  • errorobject | nullrequired

    Why the row was errored or skipped; null otherwise

    Show 2 fields of error
    • codestringrequired

      Machine-readable code

    • messagestringrequired

      One sentence for people

  • usageobjectrequired

    This row's model calls and tokens; all 0 until the row is finished

    Show 3 fields of usage
    • callsintegerrequired

      Model calls made for this row, every try included

    • input_tokensintegerrequired

      Prompt tokens over those calls

    • output_tokensintegerrequired

      Generated tokens over those calls

  • coststringrequired

    Integer nano-USD as a decimal string

    pattern ^-?[0-9]+$

Example
{
  "id": "evr_01jc5zq4m8k3v6x9t2w7r0b1e4",
  "object": "evaluation_row",
  "row": 1,
  "line": 1,
  "status": "open",
  "inputs": {},
  "generated": {},
  "verdict": {
    "label": "string",
    "score": 1.5,
    "winner": "a",
    "passes": {
      "a_first": "a",
      "b_first": "a"
    }
  },
  "passed": true,
  "reasoning": "string",
  "error": {
    "code": "string",
    "message": "string"
  },
  "usage": {
    "calls": 1,
    "input_tokens": 1,
    "output_tokens": 1
  },
  "cost": "1500000000"
}

Used by