API reference · Schema
ServingEngine
llama_cpp: GGUF chat and completion models (CPU or GPU). tei: embedding and rerank models. vllm: chat, completion and embedding models on GPUs.
Type: string — one of "llama_cpp", "tei", "vllm"
Example
"llama_cpp"