kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
b5bc0a9fcd
TensorRT-LLMs/cpp/tensorrt_llm/batch_manager/utils
History
Robin Kobus b5bc0a9fcd
chore: Add output of first token to additional generation outputs (#3205) 
- Updated the first dimension of additional output tensors to match mMaxNewTokens.
- Copy output of last context token to generation outputs.
- Adjusted the expected output size calculations in unit tests to reflect the correct maximum output length.

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
2025-04-02 20:14:16 +08:00
..
debugUtils.cpp Update TensorRT-LLM (#2873) 2025-03-11 21:13:42 +08:00
debugUtils.h Update TensorRT-LLM (#2873) 2025-03-11 21:13:42 +08:00
inflightBatchingUtils.cpp chore: Add output of first token to additional generation outputs (#3205) 2025-04-02 20:14:16 +08:00
inflightBatchingUtils.h Update TensorRT-LLM (#2873) 2025-03-11 21:13:42 +08:00
logitsThread.cpp Fix logits dtype in assert (#3038) 2025-03-25 10:35:21 +08:00
logitsThread.h Update TensorRT-LLM (#2873) 2025-03-11 21:13:42 +08:00
staticThreadPool.cpp Update TensorRT-LLM (#2873) 2025-03-11 21:13:42 +08:00
staticThreadPool.h Update TensorRT-LLM (#2873) 2025-03-11 21:13:42 +08:00