kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

[https://nvbugs/5451426][fix] Avoid torch compile on full eagle3 worker (#7245)

Browse Source 
Signed-off-by: Jin Li <59594262+liji-nv@users.noreply.github.com>
This commit is contained in:
Jin Li 2025-08-27 09:59:06 +08:00 committed by GitHub
parent 201fd257cc
commit 877e1f44d3
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 0 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
tensorrt_llm/_torch/speculative/eagle3.py
View File

@ -266,7 +266,6 @@ class Eagle3OneModelWorker(nn.Module):
self.max_draft_len = self.spec_config.max_draft_len
self.mapping = mapping
@torch.compile(options={"max-autotune": True})
def forward(self, input_ids, position_ids, hidden_states, logits,
attn_metadata, spec_metadata, draft_model):
batch_size = attn_metadata.num_seqs