TensorRT-LLMs/examples
Yechan Kim 5460d18b10
feat: trtllm-serve multimodal support (#3590)
* feat: trtllm-serve multimodal support

Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>

* remove disable argument

Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>

* remove disable

Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>

* add and separate tests and move the doc

Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>

* remove block_resue arg from serve.py

Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>

---------

Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>
Co-authored-by: Haohang Huang <31998628+symphonylyh@users.noreply.github.com>
2025-04-19 05:01:28 +08:00
..
apps doc: refactor trtllm-serve examples and doc (#3187) 2025-04-04 11:40:43 +08:00
auto_deploy feat: [TRTLLM-3510] DeepseekV3 support in AutoDeploy (#3281) 2025-04-08 21:47:57 +08:00
bert feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
bindings/executor Update TensorRT-LLM (#2582) 2024-12-16 21:50:47 -08:00
commandr feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
cpp/executor refactor: Clean up CMakeLists.txt (#3479) 2025-04-18 14:39:29 +08:00
cpp_library Update TensorRT-LLM (#1274) 2024-03-12 18:15:52 +08:00
deepseek_v3 update fp8 doc (#3647) (#3650) 2025-04-17 13:37:08 +08:00
disaggregated feat: Add support of chat completion in PD (#2985) 2025-04-11 17:53:28 +08:00
dora Update TensorRT-LLM (#2755) 2025-02-11 03:01:00 +00:00
draft_target_model feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
eagle feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
enc_dec doc: use alert formatting (#3153) 2025-03-31 07:30:52 +08:00
exaone Add EXAONE-Deep (#3054) 2025-03-26 14:24:04 +08:00
gemma feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
glm-4-9b feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
gpt feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
granite Update TensorRT-LLM (#2755) 2025-02-11 03:01:00 +00:00
infinitebench Update TensorRT-LLM (#1725) 2024-06-04 20:26:32 +08:00
internlm2 Update TensorRT-LLM (#2755) 2025-02-11 03:01:00 +00:00
language_adapter Update TensorRT-LLM (#2873) 2025-03-11 21:13:42 +08:00
llama feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
llm-api test: add kv cache event tests for disagg workers (#3602) 2025-04-18 18:30:19 +08:00
llm-eval/lm-eval-harness Update (#2978) 2025-03-23 16:39:35 +08:00
lookahead feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
mamba feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
medusa feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
mixtral Update TensorRT-LLM (#2849) 2025-03-04 18:44:00 +08:00
mllama Add Llama 4 (#3302) 2025-04-09 03:35:21 +08:00
models/contrib feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
multimodal feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
nemotron feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
nemotron_nas feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
openai_triton Update TensorRT-LLM (#2792) 2025-02-18 21:27:39 +08:00
phi feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
prompt_lookup feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
python_plugin Update TensorRT-LLM (#2755) 2025-02-11 03:01:00 +00:00
pytorch feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
quantization feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
qwen feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
qwen2audio feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
qwenvl feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
recurrentgemma feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
redrafter feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
sample_weight_stripping Update (#2978) 2025-03-23 16:39:35 +08:00
scaffolding feat: Add Dynasor-CoT in scaffolding examples. (#3501) 2025-04-18 07:48:01 +00:00
serve feat: trtllm-serve multimodal support (#3590) 2025-04-19 05:01:28 +08:00
vit Update (#2978) 2025-03-23 16:39:35 +08:00
whisper feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
constraints.txt chore: bump version to 0.20.0rc0 (#3561) 2025-04-16 11:41:21 +08:00
eval_long_context.py Feat: Variable-Beam-Width-Search (VBWS) part3 (#3338) 2025-04-08 23:51:27 +08:00
generate_checkpoint_config.py Update TensorRT-LLM (#2562) 2024-12-11 00:31:05 -08:00
generate_xgrammar_tokenizer_info.py Update TensorRT-LLM (#2783) 2025-02-13 18:40:22 +08:00
gpqa_llmapi.py feat: add qwen2 moe to torch flow; fix wrong imported KvCacheConfig in gpqa… (#3369) 2025-04-10 22:45:57 +08:00
hf_lora_convert.py Update TensorRT-LLM (#2755) 2025-02-11 03:01:00 +00:00
mmlu_llmapi.py test: Accuracy test improvement (Part 2): Incorporate mmlu to accuracy test suite (#2982) 2025-03-25 07:34:10 +08:00
mmlu.py test: Accuracy test improvement (Part 2): Incorporate mmlu to accuracy test suite (#2982) 2025-03-25 07:34:10 +08:00
run.py Update run.py for draft_target_model (#3615) 2025-04-19 01:01:50 +08:00
summarize.py feat: Integrate GPUDirect Storage (GDS) into Executor API (#3582) 2025-04-18 15:59:21 +08:00
utils.py Feat: Variable-Beam-Width-Search (VBWS) part3 (#3338) 2025-04-08 23:51:27 +08:00