mirror of
https://github.com/vllm-project/vllm.git
synced 2026-08-21 05:00:15 +00:00
51c1ee9b7c
Signed-off-by: wang.yuqi <[email protected]> Signed-off-by: wang.yuqi <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Disaggregated Prefill V1
This example contains scripts that demonstrate disaggregated prefill in the offline setting of vLLM.
Files
run.sh- A helper script that will runprefill_example.pyanddecode_example.pysequentially.- Make sure you are in the
examples/disaggregated/example_connectordirectory before runningrun.sh.
- Make sure you are in the
prefill_example.py- A script which performs prefill only, saving the KV state to thelocal_storagedirectory and the prompts tooutput.txt.decode_example.py- A script which performs decode only, loading the KV state from thelocal_storagedirectory and the prompts fromoutput.txt.