kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-13 22:18:36 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

[fix] Fix Llama 3.3 70b EAGLE (#4772)

Browse Source 
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
This commit is contained in:
Mike Iovine 2025-05-30 10:08:08 -04:00 committed by GitHub
parent 49f2f1f8eb
commit 8cb6163a57
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
tensorrt_llm/_torch/models/modeling_llama.py
View File

@ -1307,7 +1307,7 @@ class Eagle3LlamaForCausalLM(DecoderModelForCausalLM[Eagle3LlamaDraftModel,
num_tokens: int) -> Dict[str, Any]:
hidden_states = torch.empty(batch_size * num_tokens,
self.model.hidden_size_in,
self.model.hidden_size,
dtype=self.model.dtype,
device='cuda')