mirror of
https://github.com/vllm-project/vllm.git
synced 2026-07-26 00:24:33 +00:00
wang.yuqi
257af77bc2
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
860 B
860 B
Renderer APIs
Our renderer API is designed to disaggregate the render phase(preprocessing) and enable a token-in / token-out API server.
- GPU-less deployment of frontend: Allow preprocessing (tokenization, MM input processing) and postprocessing (detokenization, tool call parsing, reasoning parsing) to run without GPU.
- Disaggregated tokenization: Support use cases such as llm-d, Dynamo, and custom frontends that need to leverage vLLM's preprocessing logic without running the full inference engine.
- Tokens-in / tokens-out engine: Make the engine a pure token-in / token-out service, decoupled from request preprocessing.
API Reference
- Completions Render API (
/v1/completions/render)- Render completion requests
- Chat Completions Render API (
/v1/chat/completions/render)- Render chat completions