Skip to content
Hayonix

API reference · Schema

ServingEngine

llama_cpp: GGUF chat and completion models (CPU or GPU). tei: embedding and rerank models. vllm: chat, completion and embedding models on GPUs.

Type: string — one of "llama_cpp", "tei", "vllm"

Example
"llama_cpp"