API reference · Evaluations
List an evaluation’s rows in dataset order: verdict, reasoning, texts, cost
/v1/evaluations/{evaluation_id}/rows- TypeScript
client.evaluations.listRows()- Python
client.evaluations.list_rows()- CLI
hayonix evaluations rows
Available while the evaluation runs, so finished rows can be inspected early. Use filter to see only passing, failing, errored or inconsistent rows. Texts are shortened here; the results file has them whole.
Project API key (Authorization: Bearer sk_…), or a console session. Requires the evaluation:read permission (a key's scope, or a role that includes it).
Examples
The samples read API_KEY and API_BASE_URL from the environment (see Setup). Values are examples.
curl -sS "$API_BASE_URL/evaluations/evl_01jc5zq4m8k3v6x9t2w7r0b1e4/rows" \
-H "Authorization: Bearer $API_KEY"Request
| Header | Required | Description |
|---|---|---|
Authorization | yes | Bearer and your API key. The key implies its project. |
X-Project-Id | session only | The project to act in, for console sessions. Requests with an API key leave it out. |
Path parameters
evaluation_idstringrequiredID of the evaluation
ID with prefix evl_
Query parameters
limitinteger1–200, default 50
1–200
cursorstring`next_cursor` of the previous page
up to 200 characters
filterstring`passed`; `failed` (judged, did not pass; pairwise: B won or tie); `errored` (errors and rows never run); `inconsistent` (pairwise passes disagreed); default `all`
One of
allpassedfailederroredinconsistent
Response
200 EvaluationRowList — A page of rows
The items of this page.
ShowHide 13 fields of
dataidstringrequiredUnique ID (`evr_…`)
ID with prefix evr_
objectstringrequiredAlways `evaluation_row`: the type of this object
One of
evaluation_rowrowintegerrequired1-based data row
lineintegerrequired1-based line of the dataset file
statusstringrequired`open`: still being worked on; `judged`: has a verdict; `errored`: could not be judged (see `error`); `skipped`: never finished because the run ended early
One of
openjudgederroredskippedinputsobjectrequiredThe columns the run used (each cut to 2000 characters; the results file has them whole)
generatedobjectrequiredGenerated texts by candidate (`target`, `a`, `b`), cut the same way
ShowHide 4 fields of
verdictlabelstringMode `label`: the chosen label
scorenumberMode `grade`: the score given
winnerstringMode `pairwise`: `a`, `b` or `tie` when both judging passes agree; `inconsistent` when they disagree
One of
abtieinconsistentpassesobjectPairwise: the winner of each judging pass
ShowHide 2 fields of
passesa_firststringrequiredThe winner when A was shown to the judge first
One of
abtieb_firststringrequiredThe winner when B was shown to the judge first
One of
abtie
passedboolean | nullrequiredWhether the verdict passes; null until judged, when the evaluation defines no passing labels or score, and in mode `pairwise`
reasoningstring | nullrequiredThe judge's explanation (pairwise: one per judging pass); null until judged
errorobject | nullrequiredWhy the row was errored or skipped; null otherwise
ShowHide 2 fields of
errorcodestringrequiredMachine-readable code
messagestringrequiredOne sentence for people
usageobjectrequiredThis row's model calls and tokens; all 0 until the row is finished
ShowHide 3 fields of
usagecallsintegerrequiredModel calls made for this row, every try included
input_tokensintegerrequiredPrompt tokens over those calls
output_tokensintegerrequiredGenerated tokens over those calls
coststringrequiredInteger nano-USD as a decimal string
pattern ^-?[0-9]+$
next_cursorstring | nullrequiredPass as `cursor` for the next page; null on the last page.
Example response
{
"data": [
{
"id": "evr_01jc5zq4m8k3v6x9t2w7r0b1e4",
"object": "evaluation_row",
"row": 1,
"line": 1,
"status": "open",
"inputs": {},
"generated": {},
"verdict": {
"label": "string",
"score": 1.5,
"winner": "a",
"passes": {
"a_first": "a",
"b_first": "a"
}
},
"passed": true,
"reasoning": "string",
"error": {
"code": "string",
"message": "string"
},
"usage": {
"calls": 1,
"input_tokens": 1,
"output_tokens": 1
},
"cost": "1500000000"
}
],
"next_cursor": "string"
}Errors
Errors use the standard envelope. What this operation can answer:
| Status | Type | When |
|---|---|---|
400 | invalid_request_error | A parameter or body field is missing or not allowed; param names it. |
401 | authentication_error | The key or session is missing, revoked or expired. |
403 | permission_error | The credential lacks the permission this needs, or the organization is suspended. |
404 | not_found_error | No such resource in this project (an ID from another project is answered the same way). |
429 | rate_limit_error rate_limited | Too many requests; retry after retry-after seconds. |