kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
311 Commits 25 Branches 59 Tags 2.1 GiB
aab6214801
Commit Graph

3 Commits

Author SHA1 Message Date
Robin Kobus
b5bc0a9fcd
chore: Add output of first token to additional generation outputs (#3205) 
- Updated the first dimension of additional output tensors to match mMaxNewTokens.
- Copy output of last context token to generation outputs.
- Adjusted the expected output size calculations in unit tests to reflect the correct maximum output length.

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
 2025-04-02 20:14:16 +08:00
Aurelien Chartier
a33c595c88
Fix logits dtype in assert (#3038) 
Remove extra methods in trtGptModelInflightBatching.h. The methods were moved out of that class during a previous refactoring, but the definitions have been left behind.

Signed-off-by: Aurelien Chartier <achartier@nvidia.com>
 2025-03-25 10:35:21 +08:00
Kaiyu Xie
9b931c0f63
Update TensorRT-LLM (#2873) 2025-03-11 21:13:42 +08:00