TensorRT-LLMs/tensorrt_llm/_torch/speculative
Netanel Haber aa72d39b72
MTP and derivatives: Align sample state with trtllm sampler sample state (#5675)
This PR moves MTPSampler and derivatives to use the universal seq_slot indexing for sampling.
This is the last piece of the puzzle: After this, all of the samplers will use this format.
See: 6ee94c7
Signed-off-by: Netanel Haber <nhaber@nvidia.com>
2025-07-03 19:55:48 +02:00
..
__init__.py [TRTLLM-5000][feat] NGrams V2 (#4569) 2025-06-27 23:00:17 +08:00
draft_target.py Speculation: Draft Target in new FW (#4558) 2025-06-17 02:26:08 +08:00
drafter.py [TRTLLM-5000][feat] NGrams V2 (#4569) 2025-06-27 23:00:17 +08:00
eagle3.py MTP and derivatives: Align sample state with trtllm sampler sample state (#5675) 2025-07-03 19:55:48 +02:00
interface.py [https://nvbugspro.nvidia.com/bug/5329655] [feat] Pytorch path add spec dec param to attention op (#5146) 2025-07-02 04:54:43 -04:00
mtp.py MTP and derivatives: Align sample state with trtllm sampler sample state (#5675) 2025-07-03 19:55:48 +02:00
ngram.py [TRTLLM-5000][feat] NGrams V2 (#4569) 2025-06-27 23:00:17 +08:00
utils.py Reintroduce with perf fixes: feature: unify new_tokens format sample state to trtllm samper tokens format (#5513) 2025-06-30 11:58:59 -07:00