kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

[None][feat] Pass KvCacheRetentionConfig to torch LlmRequest (#8634)

Browse Source 
Signed-off-by: Aurelien Chartier <2567591+achartier@users.noreply.github.com>
This commit is contained in:
Aurelien Chartier 2025-10-24 06:44:34 -07:00 committed by GitHub
parent 2420918e5b
commit cdf0403c64
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
tensorrt_llm/_torch/pyexecutor/llm_request.py
View File

@ -764,7 +764,8 @@ def executor_request_to_llm_request(
cache_salt_id=executor_request.cache_salt_id,
arrival_time=getattr(executor_request, "py_arrival_time", None),
py_multimodal_data=getattr(executor_request, "py_multimodal_data",
None))
None),
kv_cache_retention_config=executor_request.kv_cache_retention_config)
if child_req_ids:
for child_id in child_req_ids:
llm_request.create_child_request(child_id)