Skip to content
Hayonix

API reference · Evaluations

Cancel an evaluation (calls in flight may finish; finished rows are kept)

POST/v1/evaluations/{evaluation_id}/cancel
API keyevaluation:updateProject
TypeScript
client.evaluations.cancel()
Python
client.evaluations.cancel()
CLI
hayonix evaluations cancel

No new call starts. The evaluation moves to cancelling, then to cancelled once its results file is written; unfinished rows become skipped and are not charged. Cancelling again returns the evaluation unchanged; one that has already ended answers 409 evaluation_not_cancellable.

Project API key (Authorization: Bearer sk_…), or a console session. Requires the evaluation:update permission (a key's scope, or a role that includes it).

Examples

The samples read API_KEY and API_BASE_URL from the environment (see Setup). Values are examples.

curl -sS -X POST "$API_BASE_URL/evaluations/evl_01jc5zq4m8k3v6x9t2w7r0b1e4/cancel" \
  -H "Authorization: Bearer $API_KEY"

Request

Headers
HeaderRequiredDescription
AuthorizationyesBearer and your API key. The key implies its project.
X-Project-Idsession onlyThe project to act in, for console sessions. Requests with an API key leave it out.

Path parameters

  • evaluation_idstringrequired

    ID of the evaluation

    ID with prefix evl_

Response

200 Evaluation — The evaluation, cancelling

  • idstringrequired

    Unique ID (`evl_…`)

    ID with prefix evl_

  • objectstringrequired

    Always `evaluation`: the type of this object

    One ofevaluation

  • project_idstringrequired

    Project it belongs to

    ID with prefix prj_

  • namestring | nullrequired

    Its name

  • modestringrequired

    `label`: one label per row; `grade`: a score per row; `pairwise`: the better of two candidates per row

    One oflabelgradepairwise

  • statusstringrequired

    validating → running → finalizing → completed | stopped | failed; cancelling → cancelled. `stopped`: ended early (credit, time limit, revoked key) with every finished row kept.

    One ofvalidatingrunningfinalizingcompletedstoppedfailedcancellingcancelled

  • stop_reasonstring | nullrequired

    Why the evaluation ended before every row was finished: cancelled, expired, insufficient_credit, spend_limit_reached, organization_suspended, api_key_revoked, billing_<reason>; null otherwise

  • dataset_file_idstringrequired

    ID of the dataset file

    ID with prefix file_

  • result_file_idstring | nullrequired

    JSONL, one line per row (purpose `evaluation_results`), once ended

  • judgeobjectrequired

    The model that gives each verdict, and its settings

    Show 4 fields of judge
    • modelstringrequired

      Model ID (`publisher/name`)

    • instructionsstringrequired

      What the judge assesses, a template over the row

    • max_tokensintegerrequired

      Most tokens per judge answer

    • temperaturenumberrequired

      The judge's sampling temperature

  • targetEvaluationCandidaterequired
    Show 6 fields of target
    • columnstring | nullrequired

      The dataset column whose text is judged; null when a model generates it

    • modelstring | nullrequired

      The model that generates the text; null when the text comes from a column

    • promptstring | nullrequired

      With `model`: the user message template; null otherwise

    • system_promptstring | nullrequired

      With `model`: the system message template; null when none was given

    • max_tokensinteger | nullrequired

      With `model`: most tokens generated per row; null otherwise

    • temperaturenumber | nullrequired

      With `model`: sampling temperature; null otherwise

  • pairwiseobject | nullrequired

    Mode `pairwise`: the two candidates; null in other modes

    Show 2 fields of pairwise
    • Show 6 fields of a
      • columnstring | nullrequired

        The dataset column whose text is judged; null when a model generates it

      • modelstring | nullrequired

        The model that generates the text; null when the text comes from a column

      • promptstring | nullrequired

        With `model`: the user message template; null otherwise

      • system_promptstring | nullrequired

        With `model`: the system message template; null when none was given

      • max_tokensinteger | nullrequired

        With `model`: most tokens generated per row; null otherwise

      • temperaturenumber | nullrequired

        With `model`: sampling temperature; null otherwise

    • Show 6 fields of b
      • columnstring | nullrequired

        The dataset column whose text is judged; null when a model generates it

      • modelstring | nullrequired

        The model that generates the text; null when the text comes from a column

      • promptstring | nullrequired

        With `model`: the user message template; null otherwise

      • system_promptstring | nullrequired

        With `model`: the system message template; null when none was given

      • max_tokensinteger | nullrequired

        With `model`: most tokens generated per row; null otherwise

      • temperaturenumber | nullrequired

        With `model`: sampling temperature; null otherwise

  • labelobject | nullrequired

    Mode `label`: its settings; null in other modes

    Show 2 fields of label
    • choicesarray of stringrequired

      The labels the judge chooses from

    • passingarray of stringrequired

      Labels that count as passing; empty when no pass rate is computed

  • gradeobject | nullrequired

    Mode `grade`: its settings; null in other modes

    Show 3 fields of grade
    • minnumberrequired

      Lowest allowed score (inclusive)

    • maxnumberrequired

      Highest allowed score (inclusive)

    • passing_atnumber | nullrequired

      Time passing (ISO 8601, UTC)

  • progressobjectrequired

    How far the run has got

    Show 6 fields of progress
    • rows_totalintegerrequired

      Rows in the dataset; 0 until validation has finished

    • rows_judgedintegerrequired

      Rows that received a verdict

    • rows_erroredintegerrequired

      Rows that could not be judged

    • rows_skippedintegerrequired

      Rows never finished because the run ended early

    • calls_completedintegerrequired

      Model calls finished so far, retries included

    • calls_plannedintegerrequired

      Rows × calls per row, before retries

  • resultsEvaluationResultsrequired

    Aggregate results so far (final once ended)

    Show 6 fields of results
    • rowsobjectrequired

      Rows by status

      Show 5 fields of rows
      • totalintegerrequired

        Rows in the dataset

      • judgedintegerrequired

        Rows that received a verdict

      • erroredintegerrequired

        Rows that could not be judged (a failed call, a missing column, no usable judge answer)

      • skippedintegerrequired

        Never finished: the run ended early

      • openintegerrequired

        Rows still being worked on

    • passedinteger | nullrequired

      Judged rows that pass; null when the evaluation defines no passing labels or score, and in mode `pairwise`

    • pass_ratenumber | nullrequired

      Passed ÷ judged, 0–1

    • labelsarray of object | nullrequired

      Mode `label`: every choice with its count, zeros included; null in other modes

      Show 2 fields of labels
      • labelstringrequired

        One of `label.choices`

      • countintegerrequired

        Judged rows given this label

    • scoresobject | nullrequired

      Mode `grade`: statistics of the scores; null in other modes

      Show 5 fields of scores
      • meannumber | nullrequired

        Mean score; null before any row is judged

      • std_devnumber | nullrequired

        Population standard deviation of the scores; null before any row is judged

      • minnumber | nullrequired

        Lowest score given; null before any row is judged

      • maxnumber | nullrequired

        Highest score given; null before any row is judged

      • histogramarray of objectrequired

        Scores over `grade.min`–`grade.max`: one bucket per whole number when there are at most 20, else 20 equal-width buckets

        Show 3 fields of histogram
        • fromnumberrequired

          Lower bound of the bucket (inclusive)

        • tonumberrequired

          Upper bound of the bucket; equal to `from` when each bucket is one whole-number score

        • countintegerrequired

          Judged rows whose score falls in the bucket

    • pairwiseobject | nullrequired

      Mode `pairwise`: how the two candidates compared; null in other modes

      Show 5 fields of pairwise
      • a_winsintegerrequired

        Pairs where both passes chose A

      • b_winsintegerrequired

        Pairs where both passes chose B

      • tiesintegerrequired

        Pairs where both passes called a tie

      • inconsistentintegerrequired

        Pairs whose two judging passes (A first, B first) disagreed

      • a_win_ratenumber | nullrequired

        A wins ÷ (A wins + B wins + ties), 0–1, over consistent pairs only; null before any consistent pair

  • usageobjectrequired

    Tokens of every model call completed so far

    Show 3 fields of usage
    • input_tokensintegerrequired

      Prompt tokens over all completed calls

    • cached_input_tokensintegerrequired

      Of `input_tokens`, those served from the prompt cache

    • output_tokensintegerrequired

      Generated tokens over all completed calls

  • coststringrequired

    Charged so far (every call, at batch prices)

    pattern ^-?[0-9]+$

  • problemsarray of objectrequired

    Why the evaluation failed

    Show 4 fields of problems
    • codestringrequired

      Machine-readable code

    • messagestringrequired

      One sentence for people

    • paramstring | nullrequired

      The request field at fault; null when not about one field

    • lineinteger | nullrequired

      1-based line of the dataset file; null when not about one line

  • created_atstring (date-time)required

    When it was created (ISO 8601, UTC)

  • running_atstring (date-time) | nullrequired

    Time running (ISO 8601, UTC)

  • finalizing_atstring (date-time) | nullrequired

    Time finalizing (ISO 8601, UTC)

  • cancelling_atstring (date-time) | nullrequired

    Time cancelling (ISO 8601, UTC)

  • ended_atstring (date-time) | nullrequired

    When it ended (null until then) (ISO 8601, UTC)

  • expires_atstring (date-time)required

    When a run still going stops

Example response
Example response
{
  "id": "evl_01jc5zq4m8k3v6x9t2w7r0b1e4",
  "object": "evaluation",
  "project_id": "prj_01jc5zq4m8k3v6x9t2w7r0b1e4",
  "name": "string",
  "mode": "label",
  "status": "validating",
  "stop_reason": "string",
  "dataset_file_id": "file_01jc5zq4m8k3v6x9t2w7r0b1e4",
  "result_file_id": "string",
  "judge": {
    "model": "string",
    "instructions": "string",
    "max_tokens": 1,
    "temperature": 1.5
  },
  "target": {
    "column": "string",
    "model": "string",
    "prompt": "string",
    "system_prompt": "string",
    "max_tokens": 1,
    "temperature": 1.5
  },
  "pairwise": {
    "a": {
      "column": "string",
      "model": "string",
      "prompt": "string",
      "system_prompt": "string",
      "max_tokens": 1,
      "temperature": 1.5
    },
    "b": {
      "column": "string",
      "model": "string",
      "prompt": "string",
      "system_prompt": "string",
      "max_tokens": 1,
      "temperature": 1.5
    }
  },
  "label": {
    "choices": [
      "string"
    ],
    "passing": [
      "string"
    ]
  },
  "grade": {
    "min": 1.5,
    "max": 1.5,
    "passing_at": 1.5
  },
  "progress": {
    "rows_total": 1,
    "rows_judged": 1,
    "rows_errored": 1,
    "rows_skipped": 1,
    "calls_completed": 1,
    "calls_planned": 1
  },
  "results": {
    "rows": {
      "total": 1,
      "judged": 1,
      "errored": 1,
      "skipped": 1,
      "open": 1
    },
    "passed": 1,
    "pass_rate": 1.5,
    "labels": [
      {
        "label": "string",
        "count": 1
      }
    ],
    "scores": {
      "mean": 1.5,
      "std_dev": 1.5,
      "min": 1.5,
      "max": 1.5,
      "histogram": [
        {
          "from": 1.5,
          "to": 1.5,
          "count": 1
        }
      ]
    },
    "pairwise": {
      "a_wins": 1,
      "b_wins": 1,
      "ties": 1,
      "inconsistent": 1,
      "a_win_rate": 1.5
    }
  },
  "usage": {
    "input_tokens": 1,
    "cached_input_tokens": 1,
    "output_tokens": 1
  },
  "cost": "1500000000",
  "problems": [
    {
      "code": "string",
      "message": "string",
      "param": "string",
      "line": 1
    }
  ],
  "created_at": "2026-01-01T00:00:00.000Z",
  "running_at": "2026-01-01T00:00:00.000Z",
  "finalizing_at": "2026-01-01T00:00:00.000Z",
  "cancelling_at": "2026-01-01T00:00:00.000Z",
  "ended_at": "2026-01-01T00:00:00.000Z",
  "expires_at": "2026-01-01T00:00:00.000Z"
}

Errors

Errors use the standard envelope. What this operation can answer:

Errors
StatusTypeWhen
400invalid_request_errorA parameter or body field is missing or not allowed; param names it.
401authentication_errorThe key or session is missing, revoked or expired.
403permission_errorThe credential lacks the permission this needs, or the organization is suspended.
404not_found_errorNo such resource in this project (an ID from another project is answered the same way).
409conflict_errorThe resource's current state does not allow this.
429rate_limit_error rate_limitedToo many requests; retry after retry-after seconds.