kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-02-05 02:31:33 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

[None][fix] Cherry-pick #10715: Disable short profile for tunable ops with MERGE strategy (#10844)

Browse Source 
Signed-off-by: Yukun He <23156053+hyukn@users.noreply.github.com>
This commit is contained in:
Yukun He 2026-01-21 13:53:04 +08:00 committed by GitHub
parent 87073d1ce4
commit d60d6ff6fd
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 4 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
tensorrt_llm/_torch/autotuner.py
View File

@ -1105,7 +1105,10 @@ class AutoTuner:
disable_short_profile = os.environ.get(
"TLLM_AUTOTUNER_DISABLE_SHORT_PROFILE", "0") == "1"
if fewer_repeat_avg_time > short_profile_threshold_ms and not disable_short_profile:
# Disable this feature for merged tuning strategy to avoid potential hang due to asymmetric tuning.
if fewer_repeat_avg_time > short_profile_threshold_ms and not disable_short_profile \
and tuning_config.distributed_tuning_strategy != DistributedTuningStrategy.MERGE:
# directly use the few repeat estimated time to avoid redundant profiling
avg_time = fewer_repeat_avg_time
else: