Skip to content
Hayonix

API reference · Evaluations

List an evaluation’s rows in dataset order: verdict, reasoning, texts, cost

GET/v1/evaluations/{evaluation_id}/rows
API keyevaluation:readProjectPaginated
TypeScript
client.evaluations.listRows()
Python
client.evaluations.list_rows()
CLI
hayonix evaluations rows

Available while the evaluation runs, so finished rows can be inspected early. Use filter to see only passing, failing, errored or inconsistent rows. Texts are shortened here; the results file has them whole.

Project API key (Authorization: Bearer sk_…), or a console session. Requires the evaluation:read permission (a key's scope, or a role that includes it).

Examples

The samples read API_KEY and API_BASE_URL from the environment (see Setup). Values are examples.

curl -sS "$API_BASE_URL/evaluations/evl_01jc5zq4m8k3v6x9t2w7r0b1e4/rows" \
  -H "Authorization: Bearer $API_KEY"

Request

Headers
HeaderRequiredDescription
AuthorizationyesBearer and your API key. The key implies its project.
X-Project-Idsession onlyThe project to act in, for console sessions. Requests with an API key leave it out.

Path parameters

  • evaluation_idstringrequired

    ID of the evaluation

    ID with prefix evl_

Query parameters

  • limitinteger

    1–200, default 50

    1–200

  • cursorstring

    `next_cursor` of the previous page

    up to 200 characters

  • filterstring

    `passed`; `failed` (judged, did not pass; pairwise: B won or tie); `errored` (errors and rows never run); `inconsistent` (pairwise passes disagreed); default `all`

    One ofallpassedfailederroredinconsistent

Response

200 EvaluationRowList — A page of rows

  • The items of this page.

    Show 13 fields of data
    • idstringrequired

      Unique ID (`evr_…`)

      ID with prefix evr_

    • objectstringrequired

      Always `evaluation_row`: the type of this object

      One ofevaluation_row

    • rowintegerrequired

      1-based data row

    • lineintegerrequired

      1-based line of the dataset file

    • statusstringrequired

      `open`: still being worked on; `judged`: has a verdict; `errored`: could not be judged (see `error`); `skipped`: never finished because the run ended early

      One ofopenjudgederroredskipped

    • inputsobjectrequired

      The columns the run used (each cut to 2000 characters; the results file has them whole)

    • generatedobjectrequired

      Generated texts by candidate (`target`, `a`, `b`), cut the same way

    • verdictEvaluationVerdictrequired
      Show 4 fields of verdict
      • labelstring

        Mode `label`: the chosen label

      • scorenumber

        Mode `grade`: the score given

      • winnerstring

        Mode `pairwise`: `a`, `b` or `tie` when both judging passes agree; `inconsistent` when they disagree

        One ofabtieinconsistent

      • passesobject

        Pairwise: the winner of each judging pass

        Show 2 fields of passes
        • a_firststringrequired

          The winner when A was shown to the judge first

          One ofabtie

        • b_firststringrequired

          The winner when B was shown to the judge first

          One ofabtie

    • passedboolean | nullrequired

      Whether the verdict passes; null until judged, when the evaluation defines no passing labels or score, and in mode `pairwise`

    • reasoningstring | nullrequired

      The judge's explanation (pairwise: one per judging pass); null until judged

    • errorobject | nullrequired

      Why the row was errored or skipped; null otherwise

      Show 2 fields of error
      • codestringrequired

        Machine-readable code

      • messagestringrequired

        One sentence for people

    • usageobjectrequired

      This row's model calls and tokens; all 0 until the row is finished

      Show 3 fields of usage
      • callsintegerrequired

        Model calls made for this row, every try included

      • input_tokensintegerrequired

        Prompt tokens over those calls

      • output_tokensintegerrequired

        Generated tokens over those calls

    • coststringrequired

      Integer nano-USD as a decimal string

      pattern ^-?[0-9]+$

  • next_cursorstring | nullrequired

    Pass as `cursor` for the next page; null on the last page.

Example response
Example response
{
  "data": [
    {
      "id": "evr_01jc5zq4m8k3v6x9t2w7r0b1e4",
      "object": "evaluation_row",
      "row": 1,
      "line": 1,
      "status": "open",
      "inputs": {},
      "generated": {},
      "verdict": {
        "label": "string",
        "score": 1.5,
        "winner": "a",
        "passes": {
          "a_first": "a",
          "b_first": "a"
        }
      },
      "passed": true,
      "reasoning": "string",
      "error": {
        "code": "string",
        "message": "string"
      },
      "usage": {
        "calls": 1,
        "input_tokens": 1,
        "output_tokens": 1
      },
      "cost": "1500000000"
    }
  ],
  "next_cursor": "string"
}

Errors

Errors use the standard envelope. What this operation can answer:

Errors
StatusTypeWhen
400invalid_request_errorA parameter or body field is missing or not allowed; param names it.
401authentication_errorThe key or session is missing, revoked or expired.
403permission_errorThe credential lacks the permission this needs, or the organization is suspended.
404not_found_errorNo such resource in this project (an ID from another project is answered the same way).
429rate_limit_error rate_limitedToo many requests; retry after retry-after seconds.