kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

[None][bug] fix export for microsoft/Phi-3-medium-128k-instruct (#10455)

Browse Source 
Signed-off-by: Tal Cherckez <127761168+tcherckez-nvidia@users.noreply.github.com>
This commit is contained in:
tcherckez-nvidia 2026-01-07 10:30:24 +02:00 committed by GitHub
parent 872210468b
commit 7e88212d24
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 4 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
tensorrt_llm/_torch/auto_deploy/models/patches/phi.py
View File

@ -70,9 +70,12 @@ def _patched_phi3_long_emb_init(
self,
):
_patched_phi3_emb_init(self)
self.ext_factors = torch.tensor(
ext_factors_tensor = torch.tensor(
self.short_factor, dtype=torch.float32, device=torch.device("cpu")
)
if hasattr(self, "ext_factors"):
delattr(self, "ext_factors")
self.register_buffer("ext_factors", ext_factors_tensor, persistent=False)
# Copied from https://huggingface.co/microsoft/Phi-3-mini-4k-instruct/blob/main/modeling_phi3.py#L151