Skip to content
Hayonix

API reference · Schema

CreateEvaluationRequest

  • namestring | null

    Its name

    up to 200 characters

  • modestringrequired

    `label`: one label per row from `label.choices`; `grade`: a number from `grade.min` to `grade.max`; `pairwise`: the better of `pairwise.a` and `pairwise.b` (judged in both orders).

    One oflabelgradepairwise

  • dataset_file_idstringrequired

    A file uploaded with purpose `evaluation` (JSONL, or CSV with a header)

    ID with prefix file_

  • Show 4 fields of judge
    • modelstringrequired

      Model ID (`publisher/name`)

      1–256 characters

    • instructionsstringrequired

      What the judge should assess, a template over the row (`{{column}}`). The texts being judged are added after it by the platform.

      1–20000 characters

    • max_tokensinteger

      Most tokens per judge answer (default 512)

      1–4096

    • temperaturenumber

      Default 0

      0–2

  • Modes `label` and `grade`

    Show 6 fields of target
    • columnstring

      Judge the text already in this dataset column

      1–128 characters

    • modelstring

      Or generate the text during the run with this model (see GET /v1/evaluations/models)

      1–256 characters

    • promptstring

      With `model`: the user message, a template over the row (`{{column}}`)

      up to 20000 characters

    • system_promptstring | null

      With `model`: an optional system message (also a template)

      up to 20000 characters

    • max_tokensinteger

      With `model`: most tokens to generate (default 512)

      1–4096

    • temperaturenumber

      With `model`: sampling temperature, 0–2 (default 0)

      0–2

  • pairwiseobject

    Mode `pairwise`: the two candidates

    Show 2 fields of pairwise
    • aany JSONrequired
      Show 6 fields of a
      • columnstring

        Judge the text already in this dataset column

        1–128 characters

      • modelstring

        Or generate the text during the run with this model (see GET /v1/evaluations/models)

        1–256 characters

      • promptstring

        With `model`: the user message, a template over the row (`{{column}}`)

        up to 20000 characters

      • system_promptstring | null

        With `model`: an optional system message (also a template)

        up to 20000 characters

      • max_tokensinteger

        With `model`: most tokens to generate (default 512)

        1–4096

      • temperaturenumber

        With `model`: sampling temperature, 0–2 (default 0)

        0–2

    • bany JSONrequired
      Show 6 fields of b
      • columnstring

        Judge the text already in this dataset column

        1–128 characters

      • modelstring

        Or generate the text during the run with this model (see GET /v1/evaluations/models)

        1–256 characters

      • promptstring

        With `model`: the user message, a template over the row (`{{column}}`)

        up to 20000 characters

      • system_promptstring | null

        With `model`: an optional system message (also a template)

        up to 20000 characters

      • max_tokensinteger

        With `model`: most tokens to generate (default 512)

        1–4096

      • temperaturenumber

        With `model`: sampling temperature, 0–2 (default 0)

        0–2

  • labelobject

    Mode `label`

    Show 2 fields of label
    • choicesarray of stringrequired

      The labels the judge chooses from (2–20); it answers with exactly one per row

      2–20 items

    • passingarray of string

      Labels that count as passing (none: no pass rate)

  • gradeobject

    Mode `grade` (whole-number bounds ask for whole-number scores)

    Show 3 fields of grade
    • minnumberrequired

      Lowest score the judge may give (inclusive)

    • maxnumberrequired

      Highest score the judge may give (inclusive)

    • passing_atnumber | null

      Scores at or above this pass (none: no pass rate)

Example
{
  "mode": "label",
  "dataset_file_id": "file_01jz8x3y4z5a6b7c8d9e0f1g2h",
  "judge": {
    "model": "qwen/qwen2.5-0.5b-instruct",
    "instructions": "Classify the sentiment of this customer review."
  },
  "target": {
    "column": "review"
  },
  "label": {
    "choices": [
      "positive",
      "negative",
      "neutral"
    ],
    "passing": [
      "positive"
    ]
  }
}

Used by