Skip to content
Hayonix

API reference

Inference API (OpenAI-compatible)

The OpenAI-compatible routes: chat completions, completions, embeddings, rerank and models, with authentication, billing, limits and errors.

The inference API follows the OpenAI API: request and response bodies have OpenAI's shape, so the official OpenAI SDKs and tools built on them work by changing the base URL and the key. It is served at INFERENCE_BASE_URL:

Environment variables
VariableValue
API_KEYsk_… (your project API key)
API_BASE_URLhttps://api.hayonix.ai/v1
INFERENCE_BASE_URLhttps://api.hayonix.ai/v1

Authenticate with a project API key that has the inference:invoke scope, as Authorization: Bearer sk_….

Routes

MethodPathWhat it does
POST/v1/chat/completionsA chat completion, optionally streamed
POST/v1/completionsA text completion, optionally streamed (chat and completion models)
POST/v1/embeddingsEmbeddings for one input or a list
POST/v1/rerankDocuments ordered by relevance to a query
GET/v1/modelsEvery model your key can use
GET/v1/models/{model}One model

Each model has a type (chat, completion, embedding or rerank), and a route accepts only models of the types it serves: a chat model answers /v1/chat/completions and /v1/completions; an embedding model answers /v1/embeddings.

Chat completions and completions

Send the OpenAI request body. The gateway reads only what it needs (model, stream, max_tokens or max_completion_tokens, n) and passes everything else (tools, response_format, sampling parameters, logprobs, image parts) to the engine unchanged. Which of those a model supports is listed in its capabilities.

jq -n --arg model "$MODEL" '{
  model: $model,
  messages: [{role: "user", content: "Name three uses for a GPU in one sentence."}],
  max_completion_tokens: 64
}' |
curl -sS "$INFERENCE_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d @- |
jq -r '"reply: " + (.choices[0].message.content // "" | gsub("^\\s+|\\s+$"; "")), "tokens: \(.usage.total_tokens)"'

With "stream": true the answer arrives as server-sent events, one data: line per chunk, ending with data: [DONE]. Add "stream_options": {"include_usage": true} to receive a final chunk with the token counts.

jq -n --arg model "$MODEL" '{
  model: $model,
  messages: [{role: "user", content: "Count from one to five."}],
  max_completion_tokens: 64,
  stream: true
}' |
curl -sS -N "$INFERENCE_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d @- |
sed -un 's/^data: //p' | grep --line-buffered -v '^\[DONE\]$' |
jq -j '.choices[0].delta.content // empty'
echo

Embeddings

jq -n --arg model "$EMBEDDING_MODEL" '{model: $model, input: ["a red bicycle", "a blue car"]}' |
curl -sS --fail-with-body "$INFERENCE_BASE_URL/embeddings" \
  -H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" -d @- |
jq -r '.data[] | "embedding \(.index): \(.embedding | length) dimensions"'

Rerank

Not part of the OpenAI API, so call it directly. The body has model, query, documents (1 to 1000 strings, or objects with a text field), optionally top_n and return_documents. Results come back best first, each with the document's index in your list and its relevance_score; return_documents: true adds the document text.

jq -n --arg model "$RERANK_MODEL" '{
  model: $model,
  query: "How do I stop paying for a pod?",
  documents: ["Terminate the pod to stop billing.", "Pods run containers.", "Volumes keep data."],
  top_n: 2
}' |
curl -sS --fail-with-body "$INFERENCE_BASE_URL/rerank" \
  -H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" -d @- |
jq -r '.results[] | "document \(.index): score \(.relevance_score)"'

Models

GET /v1/models returns OpenAI's list shape ({"object": "list", "data": [...]}, every model in one response) with extra fields on each model: type, display_name, context_length, capabilities, pricing and serving. serving is true when a replica is running the model right now; the catalog also lists models that are not served, such as base models for fine-tuning. Your project's private models (fine-tunes, dedicated endpoints) are listed only for your project.

curl -sS --fail-with-body "$INFERENCE_BASE_URL/models" -H "Authorization: Bearer $API_KEY" |
jq -r '.data[] | "\(.id)  type=\(.type)  serving=\(.serving)"'

Model IDs are publisher/name. In GET /v1/models/{model} the slash may be sent URL-encoded (%2F), as the SDKs do. Without credentials, GET /v1/models lists the public catalog only.

Billing

Requests to shared models are charged per token at the model's prices (pricing on the model). Each request first reserves credit for its worst case and settles at the token counts the engine reports when it ends; an organization without enough credit gets 402 before anything runs. A request cut off by the client is charged for its prompt and the tokens already sent. Requests to a dedicated endpoint are not charged per token; its replicas are billed by the second instead.

Limits

Requests and tokens per minute are limited per organization and model. Every response carries the x-ratelimit-* and ratelimit-* headers described in Limits and quotas; a refused request gets 429 with retry-after.

Errors

Errors use the standard envelope. The ones specific to inference:

StatuscodeWhen
400malformed_json, invalid_body, invalid_model, invalid_valueThe body is not valid JSON, or a field the gateway checks is wrong (param names it)
400model_not_supportedThe model's type does not serve this route (an embedding model on chat completions)
400engine_rejectedThe engine refused the request (for example, a parameter the model does not support)
402insufficient_credit, spend_limit_reachedNot enough credit to reserve the request
404model_not_foundNo such model, or not visible to your project
429rate_limitedOver the requests or tokens per minute
503engine_unavailable, model_unavailableNo replica can take the request right now; retry later
504timeout, engine_timeoutThe model did not answer in time

Every response carries x-request-id; quote it when you report a problem.