kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
d96c4e3379
TensorRT-LLMs/cpp/include/tensorrt_llm
History
Robin Kobus b5bc0a9fcd
chore: Add output of first token to additional generation outputs (#3205) 
- Updated the first dimension of additional output tensors to match mMaxNewTokens.
- Copy output of last context token to generation outputs.
- Adjusted the expected output size calculations in unit tests to reflect the correct maximum output length.

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
2025-04-02 20:14:16 +08:00
..
batch_manager chore: Add output of first token to additional generation outputs (#3205) 2025-04-02 20:14:16 +08:00
common feat: Variable-Beam-Width-Search (VBWS) Part2 (#3133) 2025-04-02 12:31:28 +08:00
deep_gemm [feat] open source fp8_blockscale_gemm (#3071) 2025-04-02 12:12:52 +08:00
executor chore: Ucx ip port remove mpi depend (#3101) 2025-04-02 09:42:29 +08:00
kernels Update TensorRT-LLM (#2873) 2025-03-11 21:13:42 +08:00
layers v1.2 (#3082) 2025-03-26 23:31:29 +08:00
plugins/api Update TensorRT-LLM (#2532) 2024-12-04 21:16:56 +08:00
runtime refactor: Simplify disableLookahead and improve numDecodingEngineTokens handling (#3103) 2025-04-01 18:47:31 +08:00