Skip to content
Hayonix

API reference · Schema

EvaluationEstimate

  • objectstringrequired

    Always `evaluation_estimate`: the type of this object

    One ofevaluation_estimate

  • rowsintegerrequired

    Rows in the dataset

  • calls_per_rowintegerrequired

    Model calls per row: one per generated candidate, plus one judge call (two in mode `pairwise`)

  • planned_callsintegerrequired

    Rows × calls per row, before judge retries

  • max_coststringrequired

    The most the run can cost, without judge retries

    pattern ^-?[0-9]+$

  • price_classstringrequired

    Every call is charged at the models' batch prices, as it completes

    One ofbatch

  • boundstringrequired

    How the token bound is stated

  • linesarray of objectrequired

    The bound per step of the run

    Show 7 fields of lines
    • stepstringrequired

      `target`, `a`, `b`: generating that candidate; `judge`: judging; `judge_ab` / `judge_ba`: the pairwise judging pass with A or B shown first

      One oftargetabjudgejudge_abjudge_ba

    • modelstringrequired

      Model ID (`publisher/name`)

    • billingstringrequired

      `per_token`: charged per token at batch prices; `replica_time`: a dedicated endpoint, paid by its replica time and not charged per call

      One ofper_tokenreplica_time

    • callsintegerrequired

      Calls of this step (one per row)

    • max_input_tokensintegerrequired

      Upper bound on prompt tokens over all calls of this step

    • max_output_tokensintegerrequired

      Upper bound on generated tokens over all calls of this step

    • max_coststringrequired

      Integer nano-USD as a decimal string

      pattern ^-?[0-9]+$

  • columnsarray of stringrequired

    The dataset's columns

  • columns_usedarray of stringrequired

    The columns the settings read (templates and candidate columns)

  • warningsarray of stringrequired

    Problems that will not stop the run, such as rows missing a column (those rows will be errors)

  • previewobject | nullrequired

    The first row's requests, rendered exactly as the run sends them

    Show 3 fields of preview
    • rowintegerrequired

      1-based data row shown

    • lineintegerrequired

      1-based line of the dataset file

    • requestsarray of objectrequired

      One request per step, in the order the run sends them

      Show 6 fields of requests
      • stepstringrequired

        The step the request belongs to, as in `lines`

      • modelstringrequired

        Model ID (`publisher/name`)

      • messagesarray of objectrequired

        The chat messages, with every template filled in

        Show 2 fields of messages
        • rolestringrequired

          Who the message is from

          One ofsystemuser

        • contentstringrequired

          The message text

      • max_tokensintegerrequired

        Most tokens the call may generate

      • temperaturenumberrequired

        Sampling temperature of the call

      • constrainedbooleanrequired

        Answer constrained by a JSON schema

Example
{
  "object": "evaluation_estimate",
  "rows": 1,
  "calls_per_row": 1,
  "planned_calls": 1,
  "max_cost": "1500000000",
  "price_class": "batch",
  "bound": "string",
  "lines": [
    {
      "step": "target",
      "model": "string",
      "billing": "per_token",
      "calls": 1,
      "max_input_tokens": 1,
      "max_output_tokens": 1,
      "max_cost": "1500000000"
    }
  ],
  "columns": [
    "string"
  ],
  "columns_used": [
    "string"
  ],
  "warnings": [
    "string"
  ],
  "preview": {
    "row": 1,
    "line": 1,
    "requests": [
      {
        "step": "string",
        "model": "string",
        "messages": [
          {
            "role": "system",
            "content": "string"
          }
        ],
        "max_tokens": 1,
        "temperature": 1.5,
        "constrained": true
      }
    ]
  }
}

Used by