API reference · Schema
EndpointDeployment
objectstringrequiredAlways `endpoint_deployment`: the type of this object
One of
endpoint_deploymentidstringrequiredUnique ID (`dpl_…`)
ID with prefix dpl_
endpoint_idstringrequiredID of the endpoint
ID with prefix ded_
namestringrequiredIts name
statusstringrequired`ready`: at least min_replicas ready. `degraded`: some, fewer. `failed`: stopped starting replicas after repeated failures (see last_failure); update or restart to retry.
One of
stoppedstartingreadydegradedfailedmodelobjectrequiredModel ID (`publisher/name`)
ShowHide 6 fields of
modelcatalog_model_idstring | nullrequiredID of the catalog model
repositorystringrequiredWhere the weights come from: a Hugging Face repository `<owner>/<name>`, or a `fine-tuned://` URI for a fine-tuned model
filestring | nullrequiredThe weights file or quantisation tag chosen in the repository; null when the whole repository is used
revisionstring | nullrequiredBranch, tag or commit; null for the repository's default branch
token_secret_idstring | nullrequiredID of the token secret
fine_tuned_model_idstring | nullrequiredSet when the deployment serves a fine-tuned model (then `repository` is its `fine-tuned://` URI)
`llama_cpp`: GGUF chat and completion models (CPU or GPU). `tei`: embedding and rerank models. `vllm`: chat, completion and embedding models on GPUs.
One of
llama_cppteivllmhardwareobjectrequiredThe hardware each replica runs on and its GPUs per replica
ShowHide 2 fields of
hardwareskustringrequiredHardware type code
gpu_countintegerrequiredNumber of GPUs
regionstringrequiredRegion code
engine_optionsobjectrequiredEngine settings chosen for this deployment
ShowHide 2 fields of
engine_optionscontext_lengthinteger | nullrequiredContext length in tokens per request; null: the engine's default
parallelinteger | nullrequiredRequests one replica runs at once; null: the engine's default
disk_gbintegerrequiredDisk per replica for the weights, in GB
min_replicasintegerrequiredFewest replicas kept running while the endpoint runs
max_replicasintegerrequiredMost replicas the autoscaler may run
autoscalingobjectrequiredHow the replica count follows demand
ShowHide 1 field of
autoscalingtarget_concurrencyintegerrequiredConcurrent requests per replica the autoscaler aims for: replicas = ceil(concurrency / target), between min and max
traffic_weightintegerrequiredPercentage of the endpoint's requests sent to this deployment (0–100; the weights of an endpoint's deployments sum to 100)
ShowHide 4 fields of
replicasreadyintegerrequiredReplicas taking traffic
startingintegerrequiredReplicas downloading weights or loading the model
unhealthyintegerrequiredReplicas that were ready but are failing health checks (no traffic)
drainingintegerrequiredReplicas finishing their requests before removal
last_failurestring | nullrequiredWhy the most recent replica failed to start; null when there is none. Cleared on a start of the endpoint or an update of the deployment.
created_atstring (date-time)requiredWhen it was created (ISO 8601, UTC)
updated_atstring (date-time)requiredWhen it last changed (ISO 8601, UTC)
{
"object": "endpoint_deployment",
"id": "dpl_01jc5zq4m8k3v6x9t2w7r0b1e4",
"endpoint_id": "ded_01jc5zq4m8k3v6x9t2w7r0b1e4",
"name": "string",
"status": "stopped",
"model": {
"catalog_model_id": "string",
"repository": "string",
"file": "string",
"revision": "string",
"token_secret_id": "string",
"fine_tuned_model_id": "string"
},
"engine": "llama_cpp",
"hardware": {
"sku": "string",
"gpu_count": 1
},
"region": "string",
"engine_options": {
"context_length": 1,
"parallel": 1
},
"disk_gb": 1,
"min_replicas": 1,
"max_replicas": 1,
"autoscaling": {
"target_concurrency": 1
},
"traffic_weight": 1,
"replicas": {
"ready": 1,
"starting": 1,
"unhealthy": 1,
"draining": 1
},
"last_failure": "string",
"created_at": "2026-01-01T00:00:00.000Z",
"updated_at": "2026-01-01T00:00:00.000Z"
}Used by
- An endpoint's deployments
GET /v1/endpoints/{dedicated_endpoint_id}/deployments - Add a deployment (other weights, engine or hardware). It starts with no traffic; give it a share with the traffic split.
POST /v1/endpoints/{dedicated_endpoint_id}/deployments - Get a deployment
GET /v1/endpoint-deployments/{deployment_id} - Change replica bounds or the autoscaling target (replicas are not recreated)
PATCH /v1/endpoint-deployments/{deployment_id} - Set the share of requests each deployment receives (weights sum to 100)
PUT /v1/endpoints/{dedicated_endpoint_id}/traffic