TensorRT-LLMs/examples
Suyog Gupta b0f7522c82
[AutoDeploy]feat: Add an AutoDeploy compile backend that only calls torch.compile (#4240)
* add a torch-compile backend

Signed-off-by: Suyog Gupta <41447211+suyoggupta@users.noreply.github.com>

* readme changes

Signed-off-by: Suyog Gupta <41447211+suyoggupta@users.noreply.github.com>

* plumb torch-compile through build_and_run_ad.py

Signed-off-by: Suyog Gupta <41447211+suyoggupta@users.noreply.github.com>

* plumb torch-compile through build_and_run_ad.py

Signed-off-by: Suyog Gupta <41447211+suyoggupta@users.noreply.github.com>

* plumb torch-compile through build_and_run_ad.py

Signed-off-by: Suyog Gupta <41447211+suyoggupta@users.noreply.github.com>

* add torch-cudagraph backend

Signed-off-by: Suyog Gupta <41447211+suyoggupta@users.noreply.github.com>

* update readme

Signed-off-by: Suyog Gupta <41447211+suyoggupta@users.noreply.github.com>

* update readme

Signed-off-by: Suyog Gupta <41447211+suyoggupta@users.noreply.github.com>

* further enhanced compiler backends

Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>

* further enhance readme

Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>

* better specified defaults in simple_config.py

Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>

* fix typo in simple_config.py

Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>

* updated deepseek-v3 support

Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>

* revert accidental deletion in AD Readme

Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>

---------

Signed-off-by: Suyog Gupta <41447211+suyoggupta@users.noreply.github.com>
Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>
Co-authored-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>
2025-05-16 08:38:15 +08:00
..
apps chore: Cleanup deprecated APIs from LLM-API (part 1/2) (#3732) 2025-05-07 13:20:25 +08:00
auto_deploy [AutoDeploy]feat: Add an AutoDeploy compile backend that only calls torch.compile (#4240) 2025-05-16 08:38:15 +08:00
bindings/executor Update TensorRT-LLM (#2582) 2024-12-16 21:50:47 -08:00
cpp/executor refactor: Clean up CMakeLists.txt (#3479) 2025-04-18 14:39:29 +08:00
cpp_library Update TensorRT-LLM (#1274) 2024-03-12 18:15:52 +08:00
disaggregated Breaking change: perf: Enable scheduling overlap by default (#4174) 2025-05-15 14:27:36 +08:00
dora Update TensorRT-LLM (#2755) 2025-02-11 03:01:00 +00:00
draft_target_model fix: draft target README and assertion for logits-based acceptance (#4167) 2025-05-09 16:08:47 -07:00
eagle test: Get Eagle tests working (#3593) 2025-04-20 00:50:57 +08:00
infinitebench Update TensorRT-LLM (#1725) 2024-06-04 20:26:32 +08:00
language_adapter Update TensorRT-LLM (#2873) 2025-03-11 21:13:42 +08:00
llm-api Breaking change: perf: Enable scheduling overlap by default (#4174) 2025-05-15 14:27:36 +08:00
llm-eval/lm-eval-harness Update (#2978) 2025-03-23 16:39:35 +08:00
lookahead doc: fix path after examples migration (#3814) 2025-04-24 02:36:45 +08:00
medusa feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
models refactor: use x is None instead of x == None. (#4244) 2025-05-15 20:00:04 +08:00
openai_triton Update TensorRT-LLM (#2792) 2025-02-18 21:27:39 +08:00
prompt_lookup fix: draft target README and assertion for logits-based acceptance (#4167) 2025-05-09 16:08:47 -07:00
python_plugin Update TensorRT-LLM (#2755) 2025-02-11 03:01:00 +00:00
pytorch feat: [nvbugs/5261055][nvbugs/5170160] non-invasive pipeline parallelism (#4034) 2025-05-16 04:16:53 +08:00
quantization [TRTLLM-3330][feat] Support DeepSeek-R1 W4A8 on Hopper (#4123) 2025-05-14 15:48:07 +08:00
redrafter feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
sample_weight_stripping doc: fix path after examples migration (#3814) 2025-04-24 02:36:45 +08:00
scaffolding Breaking change: perf: Enable scheduling overlap by default (#4174) 2025-05-15 14:27:36 +08:00
serve feat: enhance trtllm serve multimodal (#3757) 2025-05-15 16:16:31 -07:00
trtllm-eval [TRTLLM-4480][doc] Documentation for new accuracy test suite and trtllm-eval (#3946) 2025-05-08 19:35:23 +08:00
constraints.txt chore: bump version to 0.20.0rc3 (#4261) 2025-05-14 10:15:25 +08:00
eval_long_context.py Feat: Variable-Beam-Width-Search (VBWS) part3 (#3338) 2025-04-08 23:51:27 +08:00
generate_checkpoint_config.py Update TensorRT-LLM (#2562) 2024-12-11 00:31:05 -08:00
generate_xgrammar_tokenizer_info.py Update TensorRT-LLM (#2783) 2025-02-13 18:40:22 +08:00
hf_lora_convert.py Update TensorRT-LLM (#2755) 2025-02-11 03:01:00 +00:00
mmlu.py feat: run mmlu and summarize without engine_dir. (#4056) 2025-05-05 19:35:07 +08:00
run.py doc: fix path after examples migration (#3814) 2025-04-24 02:36:45 +08:00
summarize.py test: add INTEGRATION_TEST env var to speed up integration test (#3618) 2025-05-08 10:44:50 +08:00
utils.py feat: run mmlu and summarize without engine_dir. (#4056) 2025-05-05 19:35:07 +08:00