kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

[https://nvbugs/5474169][fix] seq_len mismatch between kv cache manager and graph attn metadata (#7606)

Browse Source 
Signed-off-by: Hui Gao <huig@nvidia.com>
Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>
This commit is contained in:
HuiGao-NV 2025-09-09 08:32:31 +08:00 committed by Yanchao Lu
parent 3cc16c2438
commit af34c9713a
1 changed files with 3 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
tensorrt_llm/_torch/pyexecutor/_util.py
View File

@ -486,6 +486,9 @@ class KvCacheCreator:
# When SWA is enabled, max_seq_len is updated inside kv_cache_manager.
if kv_cache_manager is not None:
if kv_cache_manager.max_seq_len < self._max_seq_len:
self._dummy_reqs = self._create_dummy_context_requests(
max(1, kv_cache_manager.max_seq_len - 1))
self._max_seq_len = kv_cache_manager.max_seq_len
return kv_cache_manager