kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

[TRTLLM-6342][bug] Patched incorrect starcoder tp config (#8118)

Browse Source 
Signed-off-by: greg-kwasniewski1 <213329731+greg-kwasniewski1@users.noreply.github.com>
This commit is contained in:
Grzegorz Kwasniewski 2025-10-03 00:41:59 +02:00 committed by GitHub
parent 4136942436
commit a5b59fd31d
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 4 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
tensorrt_llm/_torch/auto_deploy/models/patches/starcoder.py Normal file
View File

@ -0,0 +1,4 @@
from transformers.models.starcoder2.configuration_starcoder2 import Starcoder2Config
# Remove this patch after TRT-LLM upgrades to the HF transformers version >= 4.57
Starcoder2Config.base_model_tp_plan["layers.*.mlp.c_proj"] = "rowwise"