API reference · Schema
CreateDedicatedEndpointRequest
namestringrequiredLowercase letters, digits and hyphens. The model ID is `<org slug>/<name>`.
1–48 characters
display_namestring | nullName to show people
1–100 characters
inactivity_timeout_secondsintegerStop after this long without a request (0: never; default 3600)
0–604800
startbooleanStart serving now (default true)
deployment_namestringName of the first deployment (default `primary`)
1–48 characters
modelstring | HuggingFaceModelSource | FineTunedModelSourcerequiredModel ID (`publisher/name`)
ShowHide 2 fields of
hardwareskustringrequiredHardware type code
1–64 characters
gpu_countintegerGPUs per replica on GPU SKUs
0–8
regionstringRegion code
1–64 characters
engineany JSONengine_optionsEngineOptionsShowHide 2 fields of
engine_optionscontext_lengthintegerContext length in tokens each request may use (default: the model's context length, else the engine's default)
128–1048576
parallelintegerRequests one replica runs at once
1–256
disk_gbintegerRoom for the weights (default 10 on CPU, 200 on GPU)
1–2000
min_replicasintegerDefault 1
1–32
max_replicasintegerDefault 1
1–32
autoscalingEndpointAutoscalingShowHide 1 field of
autoscalingtarget_concurrencyintegerrequiredConcurrent requests per replica to aim for: replicas = ceil(concurrency / target), between min and max
1–1024
{
"name": "support-bot",
"model": {
"source": "huggingface",
"repository": "bartowski/SmolLM2-135M-Instruct-GGUF",
"file": "Q4_K_M",
"type": "chat"
},
"hardware": {
"sku": "cpu-2c-4gb"
},
"min_replicas": 1,
"max_replicas": 2,
"inactivity_timeout_seconds": 3600
}