API reference · Schema
CreateEndpointDeploymentRequest
namestringrequiredIts name
1–48 characters
modelstring | HuggingFaceModelSource | FineTunedModelSourcerequiredModel ID (`publisher/name`)
ShowHide 2 fields of
hardwareskustringrequiredHardware type code
1–64 characters
gpu_countintegerGPUs per replica on GPU SKUs
0–8
regionstringRegion code
1–64 characters
engineany JSONengine_optionsEngineOptionsShowHide 2 fields of
engine_optionscontext_lengthintegerContext length in tokens each request may use (default: the model's context length, else the engine's default)
128–1048576
parallelintegerRequests one replica runs at once
1–256
disk_gbintegerRoom for the weights (default 10 on CPU, 200 on GPU)
1–2000
min_replicasintegerDefault 1
1–32
max_replicasintegerDefault 1
1–32
autoscalingEndpointAutoscalingShowHide 1 field of
autoscalingtarget_concurrencyintegerrequiredConcurrent requests per replica to aim for: replicas = ceil(concurrency / target), between min and max
1–1024
Example
{
"name": "candidate",
"model": "qwen/qwen2.5-0.5b-instruct",
"hardware": {
"sku": "cpu-2c-4gb"
}
}Used by
- Add a deployment (other weights, engine or hardware). It starts with no traffic; give it a share with the traffic split.
POST /v1/endpoints/{dedicated_endpoint_id}/deployments