mirror of
https://github.com/vllm-project/vllm.git
synced 2026-08-24 22:50:15 +00:00
Yongye ZhuandClaude Opus 4.7
58c8a5eaa5
The prefill backend may be paired with flash_attn / trtllm in production — in that case the prefill backend's __init__ never runs and the prefill kernel's first call pays a 1.5–2 minute JIT cost. Add the same idempotent `warmup_compile_prefill` invocation to TokenspeedMLAImpl.__init__ (the decode-side backend, always present when tokenspeed is selected). The function dedupes by config key, so the double call is a no-op when both backends are tokenspeed. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> Signed-off-by: Yongye Zhu <[email protected]>