API reference
Inference API (OpenAI-compatible)
The OpenAI-compatible routes: chat completions, completions, embeddings, rerank and models, with authentication, billing, limits and errors.
The inference API follows the OpenAI API: request and response bodies have OpenAI's shape, so the official OpenAI SDKs and tools built on them work by changing the base URL and the key. It is served at INFERENCE_BASE_URL:
| Variable | Value |
|---|---|
API_KEY | sk_… (your project API key) |
API_BASE_URL | https://api.hayonix.ai/v1 |
INFERENCE_BASE_URL | https://api.hayonix.ai/v1 |
Authenticate with a project API key that has the inference:invoke scope, as Authorization: Bearer sk_….
Routes
| Method | Path | What it does |
|---|---|---|
POST | /v1/chat/completions | A chat completion, optionally streamed |
POST | /v1/completions | A text completion, optionally streamed (chat and completion models) |
POST | /v1/embeddings | Embeddings for one input or a list |
POST | /v1/rerank | Documents ordered by relevance to a query |
GET | /v1/models | Every model your key can use |
GET | /v1/models/{model} | One model |
Each model has a type (chat, completion, embedding or rerank), and a route accepts only models of the types it serves: a chat model answers /v1/chat/completions and /v1/completions; an embedding model answers /v1/embeddings.
Chat completions and completions
Send the OpenAI request body. The gateway reads only what it needs (model, stream, max_tokens or max_completion_tokens, n) and passes everything else (tools, response_format, sampling parameters, logprobs, image parts) to the engine unchanged. Which of those a model supports is listed in its capabilities.
jq -n --arg model "$MODEL" '{
model: $model,
messages: [{role: "user", content: "Name three uses for a GPU in one sentence."}],
max_completion_tokens: 64
}' |
curl -sS "$INFERENCE_BASE_URL/chat/completions" \
-H "Authorization: Bearer $API_KEY" \
-H "Content-Type: application/json" \
-d @- |
jq -r '"reply: " + (.choices[0].message.content // "" | gsub("^\\s+|\\s+$"; "")), "tokens: \(.usage.total_tokens)"'With "stream": true the answer arrives as server-sent events, one data: line per chunk, ending with data: [DONE]. Add "stream_options": {"include_usage": true} to receive a final chunk with the token counts.
jq -n --arg model "$MODEL" '{
model: $model,
messages: [{role: "user", content: "Count from one to five."}],
max_completion_tokens: 64,
stream: true
}' |
curl -sS -N "$INFERENCE_BASE_URL/chat/completions" \
-H "Authorization: Bearer $API_KEY" \
-H "Content-Type: application/json" \
-d @- |
sed -un 's/^data: //p' | grep --line-buffered -v '^\[DONE\]$' |
jq -j '.choices[0].delta.content // empty'
echoEmbeddings
jq -n --arg model "$EMBEDDING_MODEL" '{model: $model, input: ["a red bicycle", "a blue car"]}' |
curl -sS --fail-with-body "$INFERENCE_BASE_URL/embeddings" \
-H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" -d @- |
jq -r '.data[] | "embedding \(.index): \(.embedding | length) dimensions"'Rerank
Not part of the OpenAI API, so call it directly. The body has model, query, documents (1 to 1000 strings, or objects with a text field), optionally top_n and return_documents. Results come back best first, each with the document's index in your list and its relevance_score; return_documents: true adds the document text.
jq -n --arg model "$RERANK_MODEL" '{
model: $model,
query: "How do I stop paying for a pod?",
documents: ["Terminate the pod to stop billing.", "Pods run containers.", "Volumes keep data."],
top_n: 2
}' |
curl -sS --fail-with-body "$INFERENCE_BASE_URL/rerank" \
-H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" -d @- |
jq -r '.results[] | "document \(.index): score \(.relevance_score)"'Models
GET /v1/models returns OpenAI's list shape ({"object": "list", "data": [...]}, every model in one response) with extra fields on each model: type, display_name, context_length, capabilities, pricing and serving. serving is true when a replica is running the model right now; the catalog also lists models that are not served, such as base models for fine-tuning. Your project's private models (fine-tunes, dedicated endpoints) are listed only for your project.
curl -sS --fail-with-body "$INFERENCE_BASE_URL/models" -H "Authorization: Bearer $API_KEY" |
jq -r '.data[] | "\(.id) type=\(.type) serving=\(.serving)"'Model IDs are publisher/name. In GET /v1/models/{model} the slash may be sent URL-encoded (%2F), as the SDKs do. Without credentials, GET /v1/models lists the public catalog only.
Billing
Requests to shared models are charged per token at the model's prices (pricing on the model). Each request first reserves credit for its worst case and settles at the token counts the engine reports when it ends; an organization without enough credit gets 402 before anything runs. A request cut off by the client is charged for its prompt and the tokens already sent. Requests to a dedicated endpoint are not charged per token; its replicas are billed by the second instead.
Limits
Requests and tokens per minute are limited per organization and model. Every response carries the x-ratelimit-* and ratelimit-* headers described in Limits and quotas; a refused request gets 429 with retry-after.
Errors
Errors use the standard envelope. The ones specific to inference:
| Status | code | When |
|---|---|---|
| 400 | malformed_json, invalid_body, invalid_model, invalid_value | The body is not valid JSON, or a field the gateway checks is wrong (param names it) |
| 400 | model_not_supported | The model's type does not serve this route (an embedding model on chat completions) |
| 400 | engine_rejected | The engine refused the request (for example, a parameter the model does not support) |
| 402 | insufficient_credit, spend_limit_reached | Not enough credit to reserve the request |
| 404 | model_not_found | No such model, or not visible to your project |
| 429 | rate_limited | Over the requests or tokens per minute |
| 503 | engine_unavailable, model_unavailable | No replica can take the request right now; retry later |
| 504 | timeout, engine_timeout | The model did not answer in time |
Every response carries x-request-id; quote it when you report a problem.