API reference · Schema
UpdateServerlessEndpointRequest
namestringIts name
1–63 characters
imagestringContainer image reference
1–512 characters
registry_credential_idstring | nullID of the registry credential
ID with prefix rcr_
commandarray of stringReplaces the image's entrypoint (default: the image's own)
up to 64 items
argsarray of stringReplaces the image's default arguments (default: the image's own)
up to 256 items
Environment variables of every worker. Names starting with `WORKER_RUNTIME_` are reserved for the platform.
up to 128 items
skusarray of stringAcceptable hardware in order of preference; workers take the first with capacity
1–5 items
gpu_countintegerGPUs per worker on GPU SKUs
0–8
concurrencyintegerJobs (queue endpoints, 1–64) or requests (HTTP endpoints, 1–1000) one worker takes at once (default 1)
1–1000
disk_bytesintegerLimit on each worker container's writable disk, in bytes (default 0: no limit beyond the host's)
at least 0
Models from the model hub the worker reads (one revision per repository)
up to 8 items
ShowHide 1 field of
modelsreference_idstringrequiredA model reference of this project (POST /v1/model-references). Its files are cached on the host and mounted read-only at /model-cache/models--<owner>--<name> before the worker starts; HF_HUB_CACHE points there unless env sets it.
ID with prefix mrf_
HTTP endpoints only (required for them): where the container serves
ShowHide 2 fields of
httpportintegerrequiredThe port your container serves HTTP on
1–65535
health_pathstringPath the platform probes (GET; any 2xx or 3xx is healthy) before a worker takes requests, and to take it out of rotation when it fails (default `/health`)
pattern ^\/[A-Za-z0-9._~/?=&%-]{0,255}$
Network volumes of this project to mount into every worker (at most 4; default none). Volumes live on one host until shared storage exists, so every worker of the endpoint then runs on the host the volumes are pinned to, and waits for capacity there when it is full. Changing the list makes a new release.
up to 4 items
ShowHide 3 fields of
volumesvolume_idstringrequiredID of the volume
ID with prefix vol_
mount_pathstringrequiredAbsolute path inside every worker (not a system path such as /etc or /usr)
2–256 characters
read_onlybooleanMount read-only (default false: workers may write)
min_workersintegerDefault 0
0–100
max_workersintegerDefault 3
0–100
idle_timeout_secondsintegerAn idle worker is removed after this (default 30)
1–3600
execution_timeout_secondsintegerPer job (default 600)
1–86400
queue_ttl_secondsintegerA job not started within this expires (default 86400)
10–604800
result_ttl_secondsintegerResults are kept this long after a job finishes (default 3600)
60–604800
max_retriesintegerRe-runs after a lost worker before the job fails (default 2)
0–10
scalingServerlessScalingShowHide 2 fields of
scalingpolicystringrequiredQueue endpoints: `queue_delay` adds workers once the oldest queued job has waited `target` seconds; `requests_per_worker` keeps ceil((queued + running) / `target`) workers. HTTP endpoints: `in_flight` keeps ceil((requests in flight + waiting) / `target`) workers.
One of
queue_delayrequests_per_workerin_flighttargetintegerSeconds (queue_delay, default 5), jobs per worker (requests_per_worker, default 1) or requests in flight per worker (in_flight, default 1)
1–3600
webhook_urlstring (uri) | nullDefault completion webhook for jobs of this endpoint
up to 2048 characters
cold_start_wait_secondsintegerHTTP endpoints: a request that finds no ready worker waits this long for one to start or resume before it is answered 503 `no_worker_ready` (default 60)
1–300
drain_timeout_secondsintegerHTTP endpoints: a worker leaving rotation (scale-down, new release) gets this long to finish the requests it has in flight (default 60)
1–3600
fast_resumeServerlessFastResumeShowHide 2 fields of
fast_resumemax_paused_workersintegerIdle workers to keep paused instead of removing them, so the next request or job resumes one in a fraction of a cold start (default 0: off). Paused time is not charged.
0–10
paused_keep_secondsintegerA worker paused this long is removed (default 600)
30–86400
{
"max_workers": 5,
"image": "registry.example.com/team/captioner:1.1.0"
}Used by
- Update an endpoint. Changing the image, command, environment, SKUs, GPU count, concurrency, disk or HTTP settings creates a new release; workers are replaced progressively (busy ones finish first; an HTTP endpoint keeps old workers serving until new ones are healthy).
PATCH /v1/serverless/endpoints/{endpoint_id}