kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

[AutoDeploy] fix: disable overlap scheduler until supported (#4365)

Browse Source 
Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>
This commit is contained in:
Lucas Liebenwein 2025-05-15 19:19:30 -04:00 committed by GitHub
parent c6e2111f4e
commit 4883121477
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 3 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
tensorrt_llm/_torch/auto_deploy/shim/interface.py
View File

@ -120,3 +120,6 @@ class AutoDeployConfig(PyTorchConfig):
# gets replaced by the user provided one. We don't want that though.
f_default = self.__dataclass_fields__["model_kwargs"].default_factory()
setattr(self, "model_kwargs", {**f_default, **getattr(self, "model_kwargs")})
# TODO (https://github.com/NVIDIA/TensorRT-LLM/issues/4364) support overlap scheduler
self.disable_overlap_scheduler = True