TensorRT-LLMs/tensorrt_llm/_torch/auto_deploy
Netanel Haber 9cd8148f28
API Breaking Change + Readability: "decoder"->"sampler" (#4121)
* *decoder*->*sampler*; new_tensors_device: dict[str, torch.Tensor] -> device: SampleStateTensors

* **Breaking Change**, as it changes public interfaces, main changes:
* PyTorchConfig [consumed via LLM(pytorch_backend_config)]: Configuration parameters mixed_decoder and enable_trtllm_decoder -> sampler.
* Command-line argument --enable_trtllm_decoder becomes --enable_trtllm_sampler in examples/pytorch/quickstart_advanced.py.

---------

Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
2025-05-16 23:52:25 +08:00
..
compile [AutoDeploy]feat: Add an AutoDeploy compile backend that only calls torch.compile (#4240) 2025-05-16 08:38:15 +08:00
custom_ops feat: [AutoDeploy] update rope matcher with minor variants (Deepseek) (#3638) 2025-05-16 09:55:32 -04:00
distributed [AutoDeploy] Make all ranks agree on kv-cache size (#4007) 2025-05-02 04:07:28 +08:00
models feat:[AutoDeploy] Update MoE pattern matcher to drop expert selection logic (#3283) 2025-05-15 13:53:09 +08:00
shim API Breaking Change + Readability: "decoder"->"sampler" (#4121) 2025-05-16 23:52:25 +08:00
transformations [AutoDeploy] configurable cache resize (#4372) 2025-05-16 10:07:09 -04:00
utils feat: [AutoDeploy] update rope matcher with minor variants (Deepseek) (#3638) 2025-05-16 09:55:32 -04:00
__init__.py Update TensorRT-LLM (#2820) 2025-02-25 21:21:49 +08:00