mirror of
https://github.com/vllm-project/vllm.git
synced 2026-06-06 00:16:14 +00:00
wang.yuqi
51c1ee9b7c
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Disaggregated Prefill V1
This example contains scripts that demonstrate disaggregated prefill in the offline setting of vLLM.
Files
run.sh- A helper script that will runprefill_example.pyanddecode_example.pysequentially.- Make sure you are in the
examples/disaggregated/example_connectordirectory before runningrun.sh.
- Make sure you are in the
prefill_example.py- A script which performs prefill only, saving the KV state to thelocal_storagedirectory and the prompts tooutput.txt.decode_example.py- A script which performs decode only, loading the KV state from thelocal_storagedirectory and the prompts fromoutput.txt.