kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
3,954 Commits 25 Branches 59 Tags 2.1 GiB
8b5ededc83
Commit Graph

2155 Commits

Author SHA1 Message Date
Bo Li
8b5ededc83
[TRTLLM-9391][chore] Automatically estimate required workspace. (#9535) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
 2025-12-03 12:49:38 +08:00
Suyog Gupta
93871d52b2
[None][chore] AutoDeploy update cuda stream manager for multi-device (#9575) 
Signed-off-by: Suyog Gupta <41447211+suyoggupta@users.noreply.github.com>
 2025-12-02 20:43:14 -08:00
heyuhhh
a08eb81cce
[None][feat] Add RocketKV usage doc and e2e accuracy test on LongBenchV2 (#9572) 
Signed-off-by: yuhangh <58161490+heyuhhh@users.noreply.github.com>
 2025-12-03 11:33:46 +08:00
yufeiwu-nv
21f2ba74e8
[None][test] Remove duplicate test cases (#9623) 
Signed-off-by: yufeiwu <230315618+yufeiwu-nv@users.noreply.github.com>
 2025-12-03 10:35:26 +08:00
brb-nv
55c7023c92
[None][chore] Waive test failing on pre-merge (#9638) 
Signed-off-by: Balaram Buddharaju <169953907+brb-nv@users.noreply.github.com>
 2025-12-03 07:31:10 +08:00
Grzegorz Kwasniewski
0a7a88e74e
[TRTLLM-8946][feat] Improved heuristics to detect shardable regions (#9200) 
Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>
Signed-off-by: greg-kwasniewski1 <213329731+greg-kwasniewski1@users.noreply.github.com>
Co-authored-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>
 2025-12-02 22:08:19 +01:00
Patrice Castonguay
3991aa9c72
[https://nvbugs/5688388][fix] fix: Reducing num request in disagg test to speed up (#9598) 
Signed-off-by: Patrice Castonguay <55748270+pcastonguay@users.noreply.github.com>
 2025-12-02 12:48:53 -05:00
Neta Zmora
a560ba5546
[#9550][feat] AutoDeploy: Add NVFP4 Cutlass MoE kernels (#9551) 
Signed-off-by: Neta Zmora <96238833+nzmora-nvidia@users.noreply.github.com>
 2025-12-03 01:39:38 +08:00
Shi Xiaowei
227d42e492
[https://nvbugs/5651854][fix] Fix dist-serving perf by clearing CPU affinity (#9549) 
Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>
 2025-12-03 01:17:03 +08:00
William Zhang
2dd3ebf037
[#9150][feat] Add code for nano v3 to custom implementation in AD (#9465) 
* Why?

We would like to show an alternative to monkey-patching in AutoDeploy.

* What?

This commit builds on the existing custom model implementation for
NemotronH and adds the bits relevant for MoE layers.

Part of #9150.

Signed-off-by: William Zhang <133824995+2ez4bz@users.noreply.github.com>
 2025-12-02 08:56:44 -08:00
Mike Iovine
d5b7f0c8ad
[TRTLLM-8980][test] Clean up spec dec tests in test_llm_api_pytorch (#8889) 
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Mike Iovine <miovine@nvidia.com>
 2025-12-02 10:32:02 -05:00
Yan Chunwei
b86256eb54
[TRTLLM-9144][fix] enhance RPC robustness (#8711) 
Signed-off-by: Superjomn <328693+Superjomn@users.noreply.github.com>
Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>
Signed-off-by: Yan Chunwei <328693+Superjomn@users.noreply.github.com>
Co-authored-by: Erin Ho <14718778+hchings@users.noreply.github.com>
 2025-12-02 21:37:59 +08:00
brb-nv
be48cdf1d1
[TRTLLM-9466][test] Evaluate helix parallelism with DSV3 Lite (#9597) 
Signed-off-by: Balaram Buddharaju <169953907+brb-nv@users.noreply.github.com>
 2025-12-02 20:10:07 +08:00
Emma Qiao
4a8766c11d
[None][infra] Remove an invalid test name in waives.txt (#9620) 
Signed-off-by: qqiao <qqiao@nvidia.com>
 2025-12-02 18:05:17 +08:00
mpikulski
84a1531594
[TRTLLM-9488][feat] use FlashInfer.sampling by default (#9545) 
Signed-off-by: ixlmar <206748156+ixlmar@users.noreply.github.com>
 2025-12-02 16:29:55 +08:00
Emma Qiao
3e4f2388a9
[None][infra] Waive failed cases for main branch (#9615) 
Signed-off-by: qqiao <qqiao@nvidia.com>
 2025-12-02 15:48:27 +08:00
shuyixiong
1a2118b8fe
[https://nvbugs/5702793][fix] Fix uncontiguous tensor view (#9576) 
Signed-off-by: shuyix <219646547+shuyixiong@users.noreply.github.com>
 2025-12-02 15:41:32 +08:00
xinhe-nv
ad46d19027
[None][chore] Add failed cases into waives.txt (#9588) 
Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
 2025-12-02 14:24:11 +08:00
ruodil
4586b5f42f
[https://nvbugs/5582091][test] increase warmup times in testing for multi-gpu cases (#9578) 
Signed-off-by: Ruodi Lu <ruodil@users.noreply.github.com>
Co-authored-by: Ruodi Lu <ruodil@users.noreply.github.com>
 2025-12-02 14:22:49 +08:00
Wanli Jiang
5657a00ec0
[FMDL-1328][feat] Add support for nano-v3 and super-v3 with pytorch backend (#9261) 
Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
 2025-12-02 13:40:20 +08:00
xinhe-nv
3911d0496e
[None][fix] Waive gb200 (#9580) 
Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>
 2025-12-02 12:09:21 +08:00
JunyiXu-nv
9a6df980cd
[https://nvbugs/5703953][fix] Use random port for disagg tests (#9582) 
Signed-off-by: Junyi Xu <219237550+JunyiXu-nv@users.noreply.github.com>
 2025-12-02 11:40:14 +08:00
Iman Tabrizian
356a52edf5
[None][feat] Add support for KVCache reuse for DSv32 (#9383) 
Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>
 2025-12-02 11:14:30 +08:00
Shijie
dcf5c86720
[None][feat] Unify nvfp4 gemm backend (#8963) 
Signed-off-by: Shijie Wang <jaywan@nvidia.com>
Signed-off-by: Yukun He <23156053+hyukn@users.noreply.github.com>
Signed-off-by: Shijie <jaywan@nvidia.com>
Co-authored-by: Yukun He <23156053+hyukn@users.noreply.github.com>
 2025-12-02 11:03:51 +08:00
Eran Geva
c9771ebb99
[#9198][feat] Refactor dist ops in AutoDeploy (#9301) 
Signed-off-by: Eran Geva <19514940+MrGeva@users.noreply.github.com>
 2025-12-02 02:36:32 +08:00
Venky
639c939a4f
[TRTC-1943][feat] Env vars override support in LLM API (#9104) 
Signed-off-by: Venky Ganesh <23023424+venkywonka@users.noreply.github.com>
 2025-12-01 10:04:49 -08:00
Stefan Niebler
f155812eb0
[TRTLLM-6756][feat] Add Beam Search to TorchSampler (#8509) 
Signed-off-by: Stefan Niebler <82932102+stnie@users.noreply.github.com>
 2025-12-01 18:48:04 +01:00
Yanchao Lu
7127c4407a
[None][test] [None][test] Waive main branch test failures 12/1 (#9566) 
Signed-off-by: Yanchao Lu <yanchaol@nvidia.com>
 2025-12-01 21:54:53 +08:00
Shi Xiaowei
48b1d31895
[https://nvbugs/5651854][infra] Enable perf metrics during accuracy testing (#9140) 2025-12-01 20:15:32 +08:00
alel
4107254c82
[TRTLLM-6222][feat] Several perf opt for cuteDSL nvf4 gemm (#9428) 
Signed-off-by: Yuhan Li <51736452+liyuhannnnn@users.noreply.github.com>
 2025-12-01 18:10:45 +08:00
JadoTu
a92af27411
[None][chore] remove qwen3-next accuracy tests (#9534) 
Signed-off-by: jiant <107457950+JadoTu@users.noreply.github.com>
 2025-12-01 11:49:37 +08:00
Pengbo Wang
aa3310f64f
[https://nvbugs/5503479][fix] Temporarily lower reference accuracy to stabilize CI (#9398) 
Signed-off-by: Pengbo Wang <221450789+pengbowang-nv@users.noreply.github.com>
 2025-12-01 11:49:14 +08:00
Enwei Zhu
2e3ac3c48f
[https://nvbugs/5684703][fix] Unwaive disagg guided decoding test (#9466) 
Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
 2025-12-01 11:39:40 +08:00
Li Min
1797e91dfd
[TRTLLM-6222][feat] Extend cute_dsl_nvfp4_gemm to sm103. (#9543) 
Signed-off-by: Mindy Li <11663212+limin2021@users.noreply.github.com>
 2025-12-01 10:19:36 +08:00
heyuhhh
6e470aab72
[None] [feat] Optimize the algorithm part of RocketKV (#9333) 
Signed-off-by: yuhangh <58161490+heyuhhh@users.noreply.github.com>
 2025-12-01 09:04:09 +08:00
xxi
c12e67bb66
[TRTLLM-8958][feat] and [TRTLLM-8960]: create ConfigurableMoE and support TRTLLMGenFusedMoE as backend (#9486) 2025-12-01 08:37:07 +08:00
JunyiXu-nv
3f588198dc
[None][fix] Fix port conflict in disagg tests (#9474) 
Signed-off-by: Junyi Xu <219237550+JunyiXu-nv@users.noreply.github.com>
 2025-11-30 17:33:22 +08:00
Emma Qiao
c927ccf510
[None][infra] Wiave failed tests for main branch on 11/30 (#9555) 
Signed-off-by: qqiao <qqiao@nvidia.com>
 2025-11-30 16:13:20 +08:00
brb-nv
b77f4ffe54
[TRTLLM-5971][feat] Integrate helix parallelism (#9342) 
Signed-off-by: Balaram Buddharaju <169953907+brb-nv@users.noreply.github.com>
 2025-11-29 15:17:30 -08:00
dominicshanshan
6345074686
[None][chore] Weekly mass integration of release/1.1 -- rebase (#9522) 
Signed-off-by: yunruis <205571022+yunruis@users.noreply.github.com>
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Mike Iovine <miovine@nvidia.com>
Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>
Signed-off-by: qgai <qgai@nvidia.com>
Signed-off-by: Balaram Buddharaju <169953907+brb-nv@users.noreply.github.com>
Signed-off-by: Yan Chunwei <328693+Superjomn@users.noreply.github.com>
Signed-off-by: Junyi Xu <219237550+JunyiXu-nv@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
Signed-off-by: nv-guomingz <137257613+nv-guomingz@users.noreply.github.com>
Signed-off-by: Jin Li <59594262+liji-nv@users.noreply.github.com>
Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com>
Signed-off-by: Vincent Zhang <vinczhang@nvidia.com>
Signed-off-by: peaceh <103117813+peaceh-nv@users.noreply.github.com>
Signed-off-by: Michal Guzek <mguzek@nvidia.com>
Signed-off-by: Michal Guzek <moraxu@users.noreply.github.com>
Signed-off-by: Chang Liu (Enterprise Products) <9713593+chang-l@users.noreply.github.com>
Signed-off-by: leslie-fang25 <leslief@nvidia.com>
Signed-off-by: Shunkang <182541032+Shunkangz@users.noreply.github.co>
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
Co-authored-by: yunruis <205571022+yunruis@users.noreply.github.com>
Co-authored-by: sunnyqgg <159101675+sunnyqgg@users.noreply.github.com>
Co-authored-by: brb-nv <169953907+brb-nv@users.noreply.github.com>
Co-authored-by: Yan Chunwei <328693+Superjomn@users.noreply.github.com>
Co-authored-by: JunyiXu-nv <219237550+JunyiXu-nv@users.noreply.github.com>
Co-authored-by: Simeng Liu <109828133+SimengLiu-nv@users.noreply.github.com>
Co-authored-by: Guoming Zhang <137257613+nv-guomingz@users.noreply.github.com>
Co-authored-by: Jin Li <59594262+liji-nv@users.noreply.github.com>
Co-authored-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com>
Co-authored-by: Vincent Zhang <vcheungyi@163.com>
Co-authored-by: peaceh-nv <103117813+peaceh-nv@users.noreply.github.com>
Co-authored-by: Michal Guzek <moraxu@users.noreply.github.com>
Co-authored-by: Chang Liu <9713593+chang-l@users.noreply.github.com>
Co-authored-by: Leslie Fang <leslief@nvidia.com>
Co-authored-by: Shunkangz <182541032+Shunkangz@users.noreply.github.com>
Co-authored-by: Shunkang <182541032+Shunkangz@users.noreply.github.co>
Co-authored-by: QI JUN <22017000+QiJune@users.noreply.github.com>
 2025-11-29 21:48:48 +08:00
Grzegorz Kwasniewski
cff54fcae3
[#8948][feat] Support custom sharding config (#9143) 
Signed-off-by: greg-kwasniewski1 <213329731+greg-kwasniewski1@users.noreply.github.com>
 2025-11-29 05:28:05 +08:00
mpikulski
bc355eadf5
[TRTLLM-9488][fix] llmapi references (#9547) 
Signed-off-by: ixlmar <206748156+ixlmar@users.noreply.github.com>
 2025-11-28 08:54:05 -08:00
dominicshanshan
70efa3ac43
[None][infra] Waive failed case in pre-merge on 11/28 (#9537) 
Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>
 2025-11-28 20:53:45 +08:00
mpikulski
e5f39ec7cf
[TRTLLM-9488][feat] add 'disable_flashinfer_sampling' config option (#9454) 
Signed-off-by: ixlmar <206748156+ixlmar@users.noreply.github.com>
 2025-11-28 13:00:39 +01:00
Emma Qiao
2d7421b314
[None][infra] Waive failed cases for main branch on 11/28 (#9539) 
Signed-off-by: qqiao <qqiao@nvidia.com>
 2025-11-28 17:19:55 +08:00
Liao Lanyu
bf84d9cea1
[None][chore] add spec_decoding configs in perf benchmark scripts and fix typos (#9533) 
Signed-off-by: Lanyu Liao <lancelly@users.noreply.github.com>
Co-authored-by: Lanyu Liao <lancelly@users.noreply.github.com>
 2025-11-28 14:52:05 +08:00
yufeiwu-nv
08755a809d
[https://nvbugs/5689658][test] Fix gpu lock issue running on cluster (#9441) 
Signed-off-by: yufeiwu <230315618+yufeiwu-nv@users.noreply.github.com>
 2025-11-28 13:59:22 +08:00
Yukun He
60c43a200a
[None][fix] Fix on-disk cache and revise logger/statistics for AutoTuner. (#9211) 
Signed-off-by: Yukun He <23156053+hyukn@users.noreply.github.com>
 2025-11-28 13:32:21 +08:00
JunyiXu-nv
c87e81c1d8
[https://nvbugs/5685015][fix] Update invalid max_token test (#9435) 
Signed-off-by: Junyi Xu <219237550+JunyiXu-nv@users.noreply.github.com>
 2025-11-28 11:41:16 +08:00
Bo Li
19f3f4e520
[https://nvbugs/5637037][chore] Update waive lists. (#9386) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
Co-authored-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
 2025-11-28 10:45:22 +08:00