API reference · Evaluations
List evaluations in the project, newest first
/v1/evaluations- TypeScript
client.evaluations.list()- Python
client.evaluations.list()- CLI
hayonix evaluations list
Each evaluation carries its results: final once it has ended, computed from the rows finished so far while it runs. Filter with status.
Project API key (Authorization: Bearer sk_…), or a console session. Requires the evaluation:read permission (a key's scope, or a role that includes it).
Examples
The samples read API_KEY and API_BASE_URL from the environment (see Setup). Values are examples.
curl -sS "$API_BASE_URL/evaluations" \
-H "Authorization: Bearer $API_KEY"Request
| Header | Required | Description |
|---|---|---|
Authorization | yes | Bearer and your API key. The key implies its project. |
X-Project-Id | session only | The project to act in, for console sessions. Requests with an API key leave it out. |
Query parameters
limitinteger1–200, default 20
1–200
cursorstring`next_cursor` of the previous page
up to 200 characters
statusstringOnly evaluations in this status
One of
validatingrunningfinalizingcompletedstoppedfailedcancellingcancelled
Response
200 EvaluationList — A page of evaluations
The items of this page.
ShowHide 25 fields of
dataidstringrequiredUnique ID (`evl_…`)
ID with prefix evl_
objectstringrequiredAlways `evaluation`: the type of this object
One of
evaluationproject_idstringrequiredProject it belongs to
ID with prefix prj_
namestring | nullrequiredIts name
modestringrequired`label`: one label per row; `grade`: a score per row; `pairwise`: the better of two candidates per row
One of
labelgradepairwisestatusstringrequiredvalidating → running → finalizing → completed | stopped | failed; cancelling → cancelled. `stopped`: ended early (credit, time limit, revoked key) with every finished row kept.
One of
validatingrunningfinalizingcompletedstoppedfailedcancellingcancelledstop_reasonstring | nullrequiredWhy the evaluation ended before every row was finished: cancelled, expired, insufficient_credit, spend_limit_reached, organization_suspended, api_key_revoked, billing_<reason>; null otherwise
dataset_file_idstringrequiredID of the dataset file
ID with prefix file_
result_file_idstring | nullrequiredJSONL, one line per row (purpose `evaluation_results`), once ended
judgeobjectrequiredThe model that gives each verdict, and its settings
ShowHide 4 fields of
judgemodelstringrequiredModel ID (`publisher/name`)
instructionsstringrequiredWhat the judge assesses, a template over the row
max_tokensintegerrequiredMost tokens per judge answer
temperaturenumberrequiredThe judge's sampling temperature
ShowHide 6 fields of
targetcolumnstring | nullrequiredThe dataset column whose text is judged; null when a model generates it
modelstring | nullrequiredThe model that generates the text; null when the text comes from a column
promptstring | nullrequiredWith `model`: the user message template; null otherwise
system_promptstring | nullrequiredWith `model`: the system message template; null when none was given
max_tokensinteger | nullrequiredWith `model`: most tokens generated per row; null otherwise
temperaturenumber | nullrequiredWith `model`: sampling temperature; null otherwise
pairwiseobject | nullrequiredMode `pairwise`: the two candidates; null in other modes
ShowHide 2 fields of
pairwiseShowHide 6 fields of
acolumnstring | nullrequiredThe dataset column whose text is judged; null when a model generates it
modelstring | nullrequiredThe model that generates the text; null when the text comes from a column
promptstring | nullrequiredWith `model`: the user message template; null otherwise
system_promptstring | nullrequiredWith `model`: the system message template; null when none was given
max_tokensinteger | nullrequiredWith `model`: most tokens generated per row; null otherwise
temperaturenumber | nullrequiredWith `model`: sampling temperature; null otherwise
ShowHide 6 fields of
bcolumnstring | nullrequiredThe dataset column whose text is judged; null when a model generates it
modelstring | nullrequiredThe model that generates the text; null when the text comes from a column
promptstring | nullrequiredWith `model`: the user message template; null otherwise
system_promptstring | nullrequiredWith `model`: the system message template; null when none was given
max_tokensinteger | nullrequiredWith `model`: most tokens generated per row; null otherwise
temperaturenumber | nullrequiredWith `model`: sampling temperature; null otherwise
labelobject | nullrequiredMode `label`: its settings; null in other modes
ShowHide 2 fields of
labelchoicesarray of stringrequiredThe labels the judge chooses from
passingarray of stringrequiredLabels that count as passing; empty when no pass rate is computed
gradeobject | nullrequiredMode `grade`: its settings; null in other modes
ShowHide 3 fields of
grademinnumberrequiredLowest allowed score (inclusive)
maxnumberrequiredHighest allowed score (inclusive)
passing_atnumber | nullrequiredTime passing (ISO 8601, UTC)
progressobjectrequiredHow far the run has got
ShowHide 6 fields of
progressrows_totalintegerrequiredRows in the dataset; 0 until validation has finished
rows_judgedintegerrequiredRows that received a verdict
rows_erroredintegerrequiredRows that could not be judged
rows_skippedintegerrequiredRows never finished because the run ended early
calls_completedintegerrequiredModel calls finished so far, retries included
calls_plannedintegerrequiredRows × calls per row, before retries
Aggregate results so far (final once ended)
ShowHide 6 fields of
resultsrowsobjectrequiredRows by status
ShowHide 5 fields of
rowstotalintegerrequiredRows in the dataset
judgedintegerrequiredRows that received a verdict
erroredintegerrequiredRows that could not be judged (a failed call, a missing column, no usable judge answer)
skippedintegerrequiredNever finished: the run ended early
openintegerrequiredRows still being worked on
passedinteger | nullrequiredJudged rows that pass; null when the evaluation defines no passing labels or score, and in mode `pairwise`
pass_ratenumber | nullrequiredPassed ÷ judged, 0–1
labelsarray of object | nullrequiredMode `label`: every choice with its count, zeros included; null in other modes
ShowHide 2 fields of
labelslabelstringrequiredOne of `label.choices`
countintegerrequiredJudged rows given this label
scoresobject | nullrequiredMode `grade`: statistics of the scores; null in other modes
ShowHide 5 fields of
scoresmeannumber | nullrequiredMean score; null before any row is judged
std_devnumber | nullrequiredPopulation standard deviation of the scores; null before any row is judged
minnumber | nullrequiredLowest score given; null before any row is judged
maxnumber | nullrequiredHighest score given; null before any row is judged
histogramarray of objectrequiredScores over `grade.min`–`grade.max`: one bucket per whole number when there are at most 20, else 20 equal-width buckets
ShowHide 3 fields of
histogramfromnumberrequiredLower bound of the bucket (inclusive)
tonumberrequiredUpper bound of the bucket; equal to `from` when each bucket is one whole-number score
countintegerrequiredJudged rows whose score falls in the bucket
pairwiseobject | nullrequiredMode `pairwise`: how the two candidates compared; null in other modes
ShowHide 5 fields of
pairwisea_winsintegerrequiredPairs where both passes chose A
b_winsintegerrequiredPairs where both passes chose B
tiesintegerrequiredPairs where both passes called a tie
inconsistentintegerrequiredPairs whose two judging passes (A first, B first) disagreed
a_win_ratenumber | nullrequiredA wins ÷ (A wins + B wins + ties), 0–1, over consistent pairs only; null before any consistent pair
usageobjectrequiredTokens of every model call completed so far
ShowHide 3 fields of
usageinput_tokensintegerrequiredPrompt tokens over all completed calls
cached_input_tokensintegerrequiredOf `input_tokens`, those served from the prompt cache
output_tokensintegerrequiredGenerated tokens over all completed calls
coststringrequiredCharged so far (every call, at batch prices)
pattern ^-?[0-9]+$
problemsarray of objectrequiredWhy the evaluation failed
ShowHide 4 fields of
problemscodestringrequiredMachine-readable code
messagestringrequiredOne sentence for people
paramstring | nullrequiredThe request field at fault; null when not about one field
lineinteger | nullrequired1-based line of the dataset file; null when not about one line
created_atstring (date-time)requiredWhen it was created (ISO 8601, UTC)
running_atstring (date-time) | nullrequiredTime running (ISO 8601, UTC)
finalizing_atstring (date-time) | nullrequiredTime finalizing (ISO 8601, UTC)
cancelling_atstring (date-time) | nullrequiredTime cancelling (ISO 8601, UTC)
ended_atstring (date-time) | nullrequiredWhen it ended (null until then) (ISO 8601, UTC)
expires_atstring (date-time)requiredWhen a run still going stops
next_cursorstring | nullrequiredPass as `cursor` for the next page; null on the last page.
Example response
{
"data": [
{
"id": "evl_01jc5zq4m8k3v6x9t2w7r0b1e4",
"object": "evaluation",
"project_id": "prj_01jc5zq4m8k3v6x9t2w7r0b1e4",
"name": "string",
"mode": "label",
"status": "validating",
"stop_reason": "string",
"dataset_file_id": "file_01jc5zq4m8k3v6x9t2w7r0b1e4",
"result_file_id": "string",
"judge": {
"model": "string",
"instructions": "string",
"max_tokens": 1,
"temperature": 1.5
},
"target": {
"column": "string",
"model": "string",
"prompt": "string",
"system_prompt": "string",
"max_tokens": 1,
"temperature": 1.5
},
"pairwise": {
"a": {
"column": "string",
"model": "string",
"prompt": "string",
"system_prompt": "string",
"max_tokens": 1,
"temperature": 1.5
},
"b": {
"column": "string",
"model": "string",
"prompt": "string",
"system_prompt": "string",
"max_tokens": 1,
"temperature": 1.5
}
},
"label": {
"choices": [
"string"
],
"passing": [
"string"
]
},
"grade": {
"min": 1.5,
"max": 1.5,
"passing_at": 1.5
},
"progress": {
"rows_total": 1,
"rows_judged": 1,
"rows_errored": 1,
"rows_skipped": 1,
"calls_completed": 1,
"calls_planned": 1
},
"results": {
"rows": {
"total": 1,
"judged": 1,
"errored": 1,
"skipped": 1,
"open": 1
},
"passed": 1,
"pass_rate": 1.5,
"labels": [
{
"label": "string",
"count": 1
}
],
"scores": {
"mean": 1.5,
"std_dev": 1.5,
"min": 1.5,
"max": 1.5,
"histogram": [
{
"from": null,
"to": null,
"count": null
}
]
},
"pairwise": {
"a_wins": 1,
"b_wins": 1,
"ties": 1,
"inconsistent": 1,
"a_win_rate": 1.5
}
},
"usage": {
"input_tokens": 1,
"cached_input_tokens": 1,
"output_tokens": 1
},
"cost": "1500000000",
"problems": [
{
"code": "string",
"message": "string",
"param": "string",
"line": 1
}
],
"created_at": "2026-01-01T00:00:00.000Z",
"running_at": "2026-01-01T00:00:00.000Z",
"finalizing_at": "2026-01-01T00:00:00.000Z",
"cancelling_at": "2026-01-01T00:00:00.000Z",
"ended_at": "2026-01-01T00:00:00.000Z",
"expires_at": "2026-01-01T00:00:00.000Z"
}
],
"next_cursor": "string"
}Errors
Errors use the standard envelope. What this operation can answer:
| Status | Type | When |
|---|---|---|
400 | invalid_request_error | A parameter or body field is missing or not allowed; param names it. |
401 | authentication_error | The key or session is missing, revoked or expired. |
403 | permission_error | The credential lacks the permission this needs, or the organization is suspended. |
429 | rate_limit_error rate_limited | Too many requests; retry after retry-after seconds. |