Skip to content
Hayonix

Concepts

Limits and quotas

Organization quotas on resources, inference rate limits per model, the rate-limit headers every response carries, and what to do when you hit one.

Quotas

Each organization has quotas on how much it can run at once: GPUs, pods, serverless workers and endpoints, dedicated endpoints and their replicas, volume gigabytes, fine-tuning jobs and open batches. A new organization starts with small limits, which rise once it has made a payment that has aged for 7 days. You can ask for higher limits.

Going over a quota is refused with 429, type quota_exceeded_error and code quota_exceeded; param names the quota (for example pods). Unlike a rate limit, retrying does not help: free something up first (terminate a pod, delete an endpoint) or ask for a higher limit. The SDKs never retry a quota error.

Rate limits

The API limits requests per key. The inference API also limits requests and tokens per minute, per organization and model; batches have a separate allowance so a large batch does not slow interactive traffic.

Every response carries the state of the limit it was counted against:

HeaderMeaning
x-ratelimit-limit-requestsRequests allowed in the window
x-ratelimit-remaining-requestsRequests left in the window
x-ratelimit-reset-requestsTime until the window resets, such as 12s
x-ratelimit-limit-tokens, x-ratelimit-remaining-tokens, x-ratelimit-reset-tokensThe same for tokens (inference)
ratelimit-limit, ratelimit-remaining, ratelimit-resetRequest limit, remaining and reset in seconds
retry-afterSeconds to wait, on a refused request

A request over the limit gets 429 with type rate_limit_error. Wait for retry-after and send it again; the SDKs do this for you.

Rate-limit levels

The per-minute inference and batch limits depend on your organization's level. Every organization starts on the first level; higher levels raise every limit. A level is reached by two facts together: credit paid and kept (card payments, less refunds and open disputes; promotional credit does not count) and account age. Your level rises by itself, within minutes of a payment or of the account reaching the age, and never goes down by itself. If you need more than your level gives, ask us: a limit set for your organization wins over its level.

GET /v1/orgs/{org_id}/billing/rate-level returns your level, the limits in force and what the next level needs; the console shows the same on the Billing page.

CLI
hayonix billing rate-limits # your level, its per-minute limits and what reaches the next one