API reference · Schema
EvaluationResults
Aggregate results so far (final once ended)
rowsobjectrequiredRows by status
ShowHide 5 fields of
rowstotalintegerrequiredRows in the dataset
judgedintegerrequiredRows that received a verdict
erroredintegerrequiredRows that could not be judged (a failed call, a missing column, no usable judge answer)
skippedintegerrequiredNever finished: the run ended early
openintegerrequiredRows still being worked on
passedinteger | nullrequiredJudged rows that pass; null when the evaluation defines no passing labels or score, and in mode `pairwise`
pass_ratenumber | nullrequiredPassed ÷ judged, 0–1
labelsarray of object | nullrequiredMode `label`: every choice with its count, zeros included; null in other modes
ShowHide 2 fields of
labelslabelstringrequiredOne of `label.choices`
countintegerrequiredJudged rows given this label
scoresobject | nullrequiredMode `grade`: statistics of the scores; null in other modes
ShowHide 5 fields of
scoresmeannumber | nullrequiredMean score; null before any row is judged
std_devnumber | nullrequiredPopulation standard deviation of the scores; null before any row is judged
minnumber | nullrequiredLowest score given; null before any row is judged
maxnumber | nullrequiredHighest score given; null before any row is judged
histogramarray of objectrequiredScores over `grade.min`–`grade.max`: one bucket per whole number when there are at most 20, else 20 equal-width buckets
ShowHide 3 fields of
histogramfromnumberrequiredLower bound of the bucket (inclusive)
tonumberrequiredUpper bound of the bucket; equal to `from` when each bucket is one whole-number score
countintegerrequiredJudged rows whose score falls in the bucket
pairwiseobject | nullrequiredMode `pairwise`: how the two candidates compared; null in other modes
ShowHide 5 fields of
pairwisea_winsintegerrequiredPairs where both passes chose A
b_winsintegerrequiredPairs where both passes chose B
tiesintegerrequiredPairs where both passes called a tie
inconsistentintegerrequiredPairs whose two judging passes (A first, B first) disagreed
a_win_ratenumber | nullrequiredA wins ÷ (A wins + B wins + ties), 0–1, over consistent pairs only; null before any consistent pair
{
"rows": {
"total": 1,
"judged": 1,
"errored": 1,
"skipped": 1,
"open": 1
},
"passed": 1,
"pass_rate": 1.5,
"labels": [
{
"label": "string",
"count": 1
}
],
"scores": {
"mean": 1.5,
"std_dev": 1.5,
"min": 1.5,
"max": 1.5,
"histogram": [
{
"from": 1.5,
"to": 1.5,
"count": 1
}
]
},
"pairwise": {
"a_wins": 1,
"b_wins": 1,
"ties": 1,
"inconsistent": 1,
"a_win_rate": 1.5
}
}