kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
188 Commits 25 Branches 59 Tags 2.1 GiB
94dd456bd0
Commit Graph

2 Commits

Author SHA1 Message Date
Robin Kobus
94dd456bd0
refactor: Remove speculative decoding parameters from stateful decoders (#3024) 
Simplify StatefulGptDecoderBatched constructor:
  - Remove speculative decoding mode parameter
  - Initialize with default mode=None
  - Update GptSession class accordingly

Simplify setup method signatures in StatefulGptDecoder and StatefulGptDecoderBatched:
  - Remove maxTokensPerStep parameter
  - Initialize decoders with default maxTokensPerStep=1
  - Update GptSession class accordingly

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
 2025-03-26 20:16:26 +08:00
Kaiyu Xie
3aa6b11d13
Update TensorRT-LLM (#2936) 
* Update TensorRT-LLM

---------

Co-authored-by: changcui <cuichang147@gmail.com>
 2025-03-18 21:25:19 +08:00