kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-02-01 00:31:24 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
1,917 Commits 28 Branches 64 Tags 2 GiB
a433ebad2b
Commit Graph

475 Commits

Author SHA1 Message Date
bhsueh_NV
2e14c8f443
[Fix][Chore][Qwen3] fix bug of using fp4 on sm120 (#6065) 
Signed-off-by: bhsueh <11360707+byshiue@users.noreply.github.com>
 2025-07-20 10:25:25 +08:00
Ziyi Xiong
66030ef815
[TRTLLM-6452][feat]: Two-model engine KV cache reuse support (#6133) 
Signed-off-by: ziyixiong-nv <fxiong@nvidia.com>
Signed-off-by: ziyixiong-nv <219238287+ziyixiong-nv@users.noreply.github.com>
 2025-07-19 13:17:15 +08:00
wili
82d3587bb8
[refactor] Unify name of NGram speculative decoding (#5937) 
Signed-off-by: wili-65535 <wili-65535@users.noreply.github.com>
Co-authored-by: wili-65535 <wili-65535@users.noreply.github.com>
 2025-07-19 12:59:57 +08:00
xiaoqi
28858c8711
feat(eagle3):support qwen3 dense model (#5879) 
Signed-off-by: xq25478 <xq25478@qq.com>
 2025-07-19 01:24:32 +08:00
Bo Deng
2c6fa145ee
[TRTLLM-6471] Infra: unwaive nixl tests and some disagg-serve tests (#6095) 
Signed-off-by: Bo Deng <deemod@nvidia.com>
 2025-07-19 00:48:44 +08:00
Emma Qiao
77acb4f753
[Infra] - Waive failed tests in post-merge (#6176) 
Signed-off-by: qqiao <qqiao@nvidia.com>
 2025-07-18 17:34:34 +08:00
Zhenhuan Chen
992b273045
[https://nvbugs/5387375] fix(scaffolding): fix scaffolding aime test in test_e2e (#6140) 
Signed-off-by: Zhenhuan Chen <chenzhh3671@gmail.com>
 2025-07-18 10:34:37 +08:00
Iman Tabrizian
b75e53ab69
Revert "feat: nanobind bindings (#5961)" (#6160) 
Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>
 2025-07-18 10:12:54 +08:00
2ez4bz
8480c120b1
[fix] Fix Mistral3VLM weight-loading & enable in pre-merge (#6105) 
Signed-off-by: William Zhang <133824995+2ez4bz@users.noreply.github.com>
 2025-07-17 11:04:17 -07:00
Linda
5bff317abf
feat: nanobind bindings (#5961) 
Signed-off-by: Linda-Stadter <57756729+Linda-Stadter@users.noreply.github.com>
 2025-07-17 22:42:52 +08:00
Yi Zhang
a718486900
fix: Fix DeepSeek R1 CI (#6129) 
Signed-off-by: Yi Zhang <187001205+yizhang-nv@users.noreply.github.com>
 2025-07-17 18:24:49 +08:00
Chuang Zhu
44c70c88f9
chore:[BREAKING CHANGE] use cacheTransceiverConfig as knobs for disagg service (#5234) 
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
 2025-07-17 17:42:07 +08:00
Iman Tabrizian
d4d21a106e
[fix] Release slots with spec decode + disagg (#5975) (#6032) 
Signed-off-by: Iman Tabrizian <itabrizian@nvidia.com>
Signed-off-by: Iman Tabrizian <10105175+Tabrizian@users.noreply.github.com>
Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>
 2025-07-17 12:58:18 +08:00
chenfeiz0326
fe070a0168
test: Update Llama4 Scout FP4 & FP8 accuracy tests (#5901) 
Signed-off-by: Chenfei Zhang <chenfeiz@nvidia.com>
 2025-07-17 09:41:18 +08:00
Wanli Jiang
2d2b8bae32
feat: TRTLLM-5574 Add phi-4-multimodal pytorch-backend support (#5644) 
Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
 2025-07-17 06:30:58 +08:00
qixiang-99
e09e409dfb
Fix: Enhance ModelConfig for kv cache size calculations (#5868) 
Signed-off-by: qixiang-99 <203170375+qixiang-99@users.noreply.github.com>
 2025-07-16 14:41:31 -07:00
Emma Qiao
e30d7bec38
[Infra] - Waive failed cases in post-merge on main (#6096) 
Signed-off-by: qqiao <qqiao@nvidia.com>
 2025-07-16 22:41:18 +08:00
Ivy Zhang
dda91b5117
tests: add QA test cases (#5959) 
Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com>
 2025-07-16 16:14:25 +08:00
Ivy Zhang
763012a88a
[nvbug/5359218][tests] add test llm api test case on lookahead with chunked prefill (#6051) 
Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com>
 2025-07-16 16:04:08 +08:00
peaceh-nv
f5f31beee1
feat: Add deepseek-lite tests for RTX pro 6000 (#5903) 
Signed-off-by: peaceh <103117813+peaceh-nv@users.noreply.github.com>
 2025-07-16 15:51:45 +08:00
Wanli Jiang
8679a058a3
fix: Unable to load phi4-model with tp_size>1 (#5962) 
Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
 2025-07-16 11:39:41 +08:00
brb-nv
9214ac662a
test: Add regression tests for Gemma3 VLM (#6033) 
Signed-off-by: Balaram Buddharaju <169953907+brb-nv@users.noreply.github.com>
 2025-07-15 11:37:56 -07:00
Fanrong Li
7a1af1c738
Cherry-pick https://github.com/NVIDIA/TensorRT-LLM/pull/5947 (#5989) 
Signed-off-by: Fanrong Li <23290157+lfr-0531@users.noreply.github.com>
 2025-07-16 01:33:12 +09:00
MinaHuai
9ebc3ab9c4
[nvbugs/5385972][nvbugs/5387423][Fix] Minor fix for llava_next/llava_onevision (#5998) 
Signed-off-by: Mina Huai <121143971+MinaHuai@users.noreply.github.com>
 2025-07-15 10:01:35 -04:00
ruodil
2a147c4d01
test: add llama_v3.3_70b_cases in perf test (#6035) 
Signed-off-by: ruodil <200874449+ruodil@users.noreply.github.com>
 2025-07-15 17:53:59 +10:00
ixlmar
f225f5cd2e
[nvbugs-5318143] fix: restrict PyTorch memory usage to avoid OOMs (#5964) 
Signed-off-by: ixlmar <206748156+ixlmar@users.noreply.github.com>
 2025-07-15 06:49:42 +08:00
brb-nv
1a2d96919c
feat: Update Gemma3 Vision Encoder (#5973) 
Signed-off-by: Balaram Buddharaju <169953907+brb-nv@users.noreply.github.com>
 2025-07-14 22:38:10 +08:00
Zhenhuan Chen
30608a5e6d [https://nvbugs/5355316] fix: update torch.compile option to fix triton store_cubin error (#5865) 
Signed-off-by: Zhenhuan Chen <chenzhh3671@gmail.com>
 2025-07-14 17:17:30 +08:00
ruodil
347520494b test: remove duplicate cases in perf sanity test (#5870) 
Signed-off-by: ruodil <200874449+ruodil@users.noreply.github.com>
 2025-07-14 17:17:30 +08:00
Bo Li
6d79559f3e fix: [https://nvbugs/5351130][https://nvbugs/5333654] Unwaive for bug 5351130 and 5333654. (#5821) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
 2025-07-14 17:17:30 +08:00
Bo Li
2991cf4b80 fix: [https://nvbugspro.nvidia.com/bug/5345215] Unwaive for bug 5345215. (#5606) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
 2025-07-14 17:17:30 +08:00
Pengyun Lin
6992616c1f [nvbug 5004744][fix] rewrite completion API to avoid repetitive tokens (#5201) 
Signed-off-by: Pengyun Lin <81065165+LinPoly@users.noreply.github.com>
 2025-07-14 17:17:30 +08:00
ruodil
278a1a7df3 test: fix some test failure and add llama_nemotron models in perf sanity test, add more torch cases (#5693) 
Signed-off-by: ruodil <200874449+ruodil@users.noreply.github.com>
Co-authored-by: Larry <197874197+LarryXFly@users.noreply.github.com>
 2025-07-14 17:17:30 +08:00
Iman Tabrizian
c8874a7f94 [nvbug/5337601][fix] Fix disagg + speculative decoding (#5558) 
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>
Co-authored-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
 2025-07-14 17:17:30 +08:00
Yi Zhang
e5e87ecf34 test: Move some of the test from post merge to pre-merge, update dgx b200 test case (#5640) 
Signed-off-by: Yi Zhang <187001205+yizhang-nv@users.noreply.github.com>
 2025-07-14 17:17:30 +08:00
Yan Chunwei
9c673e9707
[TRTLLM-6160] chore: add sampling examples for pytorch (#5951) 
Signed-off-by: Superjomn <328693+Superjomn@users.noreply.github.com>
 2025-07-14 15:28:32 +09:00
Yan Chunwei
c30eead09f
[TRTLLM-6164][TRTLLM-6165] chore: add runtime example for pytorch (#5956) 
Signed-off-by: Superjomn <328693+Superjomn@users.noreply.github.com>
 2025-07-14 14:09:39 +08:00
Thor Johnsen
041f1fa513
[TRTLLM-6264] Fix flaky test_e2e.py::test_openai_lora (#5885) 
Signed-off-by: thorjohnsen <41591019+thorjohnsen@users.noreply.github.com>
 2025-07-11 16:20:41 -07:00
xinhe-nv
509363d858
tests: update sanity tests & fix tests (#5906) 
Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>
 2025-07-11 19:48:19 +10:00
brb-nv
0385f89abc
test: Fix Gemma3 unit tests due to transformers upgrade (#5921) 
Signed-off-by: Balaram Buddharaju <169953907+brb-nv@users.noreply.github.com>
 2025-07-10 17:24:10 -07:00
2ez4bz
c19840235d
[fix] Fix mistral unit tests due to transformers upgrade (#5904) 
Signed-off-by: William Zhang <133824995+2ez4bz@users.noreply.github.com>
 2025-07-10 10:45:27 -07:00
Yiqing Yan
3aa53ec36c
[None] - Waive L0 tests (#5915) 
Signed-off-by: Yiqing Yan <yiqingy@nvidia.com>
 2025-07-10 18:33:17 +08:00
Enwei Zhu
055c4a9fe6
[NvBug 5370718, 5371538] fix: Fix incremental detokenization (#5825) 
Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
 2025-07-10 16:30:00 +08:00
Anthony Chang
7d21b55b5a
[feat] Add TRTLLM MoE nvfp4 cubins for mid-high concurrency; attention_dp for TRTLLM MoE (#5723) 
Signed-off-by: Anthony Chang <27950904+rosenrodt@users.noreply.github.com>
 2025-07-10 14:06:50 +08:00
peaceh-nv
76c3a12bcb
[fix] WAR to fix the illegal memory access issue in moe gemm on SM120 (#5636) 
Signed-off-by: peaceh <103117813+peaceh-nv@users.noreply.github.com>
 2025-07-10 09:20:30 +08:00
2ez4bz
87fe44fd29
feat(models): Mistral3.1 VLM pytorch backend support (#5529) 
Signed-off-by: William Zhang <133824995+2ez4bz@users.noreply.github.com>
 2025-07-09 13:17:40 -07:00
DylanChen-NV
74dca0aa7b
[NVBUG-5304516/5319741]Qwen2.5VL FP8 support (#5029) 
Signed-off-by: Dylan Chen <191843203+DylanChen-NV@users.noreply.github.com>
 2025-07-09 23:16:42 +08:00
Bo Li
9d894bc0cb
fix: [https://nvbugspro.nvidia.com/bug/5375656] Unwaive for bug 5375656. (#5842) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
 2025-07-09 10:17:05 +08:00
Venky
e27215ca03
test: Validate and add accuracy& perf tests for Ministral-8B-Instruct[-FP8](pytorch only) (#5654) 
Signed-off-by: Venky Ganesh <23023424+venkywonka@users.noreply.github.com>
 2025-07-08 18:16:21 -07:00
xavier-nvidia
b6013da198
Fix GEMM+AR fusion on blackwell (#5563) 
Signed-off-by: xsimmons <xsimmons@nvidia.com>
 2025-07-09 08:48:47 +08:00