Concepts
Limits and quotas
Organization quotas on resources, inference rate limits per model, the rate-limit headers every response carries, and what to do when you hit one.
Quotas
Each organization has quotas on how much it can run at once: GPUs, pods, serverless workers and endpoints, dedicated endpoints and their replicas, volume gigabytes, fine-tuning jobs and open batches. A new organization starts with small limits, which rise once it has made a payment that has aged for 7 days. You can ask for higher limits.
Going over a quota is refused with 429, type quota_exceeded_error and code quota_exceeded; param names the quota (for example pods). Unlike a rate limit, retrying does not help: free something up first (terminate a pod, delete an endpoint) or ask for a higher limit. The SDKs never retry a quota error.
Rate limits
The API limits requests per key. The inference API also limits requests and tokens per minute, per organization and model; batches have a separate allowance so a large batch does not slow interactive traffic.
Every response carries the state of the limit it was counted against:
| Header | Meaning |
|---|---|
x-ratelimit-limit-requests | Requests allowed in the window |
x-ratelimit-remaining-requests | Requests left in the window |
x-ratelimit-reset-requests | Time until the window resets, such as 12s |
x-ratelimit-limit-tokens, x-ratelimit-remaining-tokens, x-ratelimit-reset-tokens | The same for tokens (inference) |
ratelimit-limit, ratelimit-remaining, ratelimit-reset | Request limit, remaining and reset in seconds |
retry-after | Seconds to wait, on a refused request |
A request over the limit gets 429 with type rate_limit_error. Wait for retry-after and send it again; the SDKs do this for you.
Rate-limit levels
The per-minute inference and batch limits depend on your organization's level. Every organization starts on the first level; higher levels raise every limit. A level is reached by two facts together: credit paid and kept (card payments, less refunds and open disputes; promotional credit does not count) and account age. Your level rises by itself, within minutes of a payment or of the account reaching the age, and never goes down by itself. If you need more than your level gives, ask us: a limit set for your organization wins over its level.
GET /v1/orgs/{org_id}/billing/rate-level returns your level, the limits in force and what the next level needs; the console shows the same on the Billing page.
hayonix billing rate-limits # your level, its per-minute limits and what reaches the next one