kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-02-12 22:14:03 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
4,611 Commits 28 Branches 67 Tags 2.1 GiB
bf16fbd86c
Commit Graph

22 Commits

Author SHA1 Message Date
mpikulski
bf7998f1b8
[TRTLLM-9522][test] cover LLM API multi_modal_embeddings (#9963) 
Signed-off-by: ixlmar <206748156+ixlmar@users.noreply.github.com>
 2026-01-12 11:38:22 +01:00
William Zhang
c0ae6bbdbe
[None][feat] EPD for Qwen3 VL (#10470) 
* Why?

We would like to support EPD disaggregated serving for Qwen3 VL.

* What?

This commit adds such support, and extends existing unit tests for
correctness checks.

Some minor (protected) interface changes had to be made to the
weight mapper as a side-effect.

Signed-off-by: William Zhang <133824995+2ez4bz@users.noreply.github.com>
 2026-01-08 06:45:54 -05:00
William Zhang
a6a88985cf
[TRTLLM-9409][feat] Pass MRoPE tensors for EPD disagg (#9758) 
* Why?

Certain VLMs like the Qwen family need more than just the multimodal
embeddings in the language model, and need MRoPE position IDs and
deltas. Prior to this commit, only the embeddings could be communicated
from the encoder worker to the prefill worker.

* What?

This commit extends the `DisaggregatedParams` to include the MRoPE
information. It also adjusts several pieces of code required to
communicate that between E, P and D workers.

Closes TRTLLM-9409.

Signed-off-by: William Zhang <133824995+2ez4bz@users.noreply.github.com>
 2025-12-22 06:32:49 -05:00
Simeng Liu
f21e2b3329
[TRTLLM-9601][feat] Expose mmKeys for multimodal to integrate with dynamo. (#9604) 
Signed-off-by: SimengLiu-nv <simengl@nvidia.com>
 2025-12-15 08:42:30 +08:00
Yechan Kim
67208f1512
[None][fix] InputProcessor config naming convention fix (#8705) 
Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>
 2025-11-03 22:29:21 -08:00
Chang Liu
f57dc01e6f
[https://nvbugs/5625380][chore] Remove multimodal related fields from decoder llm input (#8846) 2025-11-02 17:44:08 -08:00
Patrice Castonguay
afa75c9494
[https://nvbugs/5614506][chore] Adding e+p+d e2e test (#8801) 
Signed-off-by: Patrice Castonguay <55748270+pcastonguay@users.noreply.github.com>
 2025-10-31 09:52:42 -07:00
Yechan Kim
a6017f6266
[https://nvbugs/5608723][fix] Use local data on multimodal tests and unwaive tests (#8673) 
Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>
 2025-10-28 09:20:02 +09:00
Yechan Kim
2d86d6be40
[TRTLLM-8737][feat] Support media_io_kwargs on trtllm-serve (#8528) 
Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>
 2025-10-24 12:53:40 -04:00
Chang Liu
334e2cab0d
[https://nvbugs/5542867][fix] Fix the non-determinism issue in the mm_encoder test (#8033) 
Signed-off-by: Chang Liu (Enterprise Products) <9713593+chang-l@users.noreply.github.com>
 2025-09-29 09:45:16 -07:00
QI JUN
4c0f8482f1
[None][ci] Waive test_mm_encoder_standalone.py::test_multi_request_batch_chat[llava-v1.6-mistral-7b-hf] (#8010) 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
 2025-09-26 11:07:54 +08:00
Chang Liu
998857bcde
[TRTLLM-7328][feat] E-PD Disagg Support via llmapi (3/N) (#7577) 
Signed-off-by: Chang Liu (Enterprise Products) <9713593+chang-l@users.noreply.github.com>
 2025-09-22 19:07:18 -07:00
Yechan Kim
f77aca9f2c
[TRTLLM-7385][feat] Optimize Qwen2/2.5-VL performance (#7250) 
Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>
 2025-09-22 03:40:02 -07:00
Chang Liu
2e317a7db6
[https://nvbugs/5520490][fix] Fix intermittent test failures by avoiding external web data pulls (#7879) 
Signed-off-by: Chang Liu (Enterprise Products) <9713593+chang-l@users.noreply.github.com>
 2025-09-19 17:24:13 -07:00
Chang Liu
47e37755a3
[TRTLLM-6903][feat] Support chunked prefill for multimodal models (#6843) 
Signed-off-by: Chang Liu (Enterprise Products) <9713593+chang-l@users.noreply.github.com>
 2025-09-14 20:10:10 -07:00
Chang Liu
4a1e13897f
[None][feat] Update multimodal utility get_num_tokens_per_image for better generalization (#7544) 
Signed-off-by: Chang Liu (Enterprise Products) <9713593+chang-l@users.noreply.github.com>
 2025-09-08 07:42:46 -04:00
Chang Liu
99b98f1374
[TRTLLM-7440][fix] Split fused_input_embed to separate out host sync (#7280) 
Signed-off-by: Chang Liu (Enterprise Products) <9713593+chang-l@users.noreply.github.com>
 2025-09-06 23:11:39 -04:00
Chang Liu
08a0e06621
[TRTLLM-7410][feat] Support hashing and KV cache reuse for videos (#7360) 
Signed-off-by: Chang Liu (Enterprise Products) <9713593+chang-l@users.noreply.github.com>
Signed-off-by: Chang Liu <9713593+chang-l@users.noreply.github.com>
 2025-09-04 14:39:23 -04:00
Chang Liu
ce53832610
[TRTLLM-7326][feat] Add standalone multimodal encoder (#6743) 
Signed-off-by: Chang Liu <9713593+chang-l@users.noreply.github.com>
Signed-off-by: Chang Liu (Enterprise Products) <9713593+chang-l@users.noreply.github.com>
 2025-08-19 21:42:50 -07:00
Yechan Kim
60073a7ad9
[None][feat] Support SharedTensor on MultimodalParams (#6254) 
Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>
 2025-08-10 17:48:24 -07:00
Chang Liu
b4065d8ca6
[TRTLLM-6654][feat] Add support for external multimodal embeddings (#6263) 
Signed-off-by: Chang Liu <9713593+chang-l@users.noreply.github.com>
 2025-07-30 10:00:15 -04:00
Chang Liu
7381f1dba7
[TRTLLM-5059][feat] Add KV cache reuse support for multimodal models (#5444) 
Only supports qwen in this PR
 2025-07-21 16:11:58 -07:00