kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-24 20:52:48 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
27afcb9928
TensorRT-LLMs/tensorrt_llm/_torch/speculative
History
Netanel Haber 9cd8148f28
API Breaking Change + Readability: "decoder"->"sampler" (#4121) 
* *decoder*->*sampler*; new_tensors_device: dict[str, torch.Tensor] -> device: SampleStateTensors

* **Breaking Change**, as it changes public interfaces, main changes:
* PyTorchConfig [consumed via LLM(pytorch_backend_config)]: Configuration parameters mixed_decoder and enable_trtllm_decoder -> sampler.
* Command-line argument --enable_trtllm_decoder becomes --enable_trtllm_sampler in examples/pytorch/quickstart_advanced.py.

---------

Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
2025-05-16 23:52:25 +08:00
..
__init__.py Add initial EAGLE-3 implementation (#3035) 2025-03-29 22:31:24 +08:00
eagle3.py API Breaking Change + Readability: "decoder"->"sampler" (#4121) 2025-05-16 23:52:25 +08:00
interface.py API Breaking Change + Readability: "decoder"->"sampler" (#4121) 2025-05-16 23:52:25 +08:00
mtp.py API Breaking Change + Readability: "decoder"->"sampler" (#4121) 2025-05-16 23:52:25 +08:00
utils.py API Breaking Change + Readability: "decoder"->"sampler" (#4121) 2025-05-16 23:52:25 +08:00