kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
2,492 Commits 25 Branches 59 Tags 2.1 GiB
ee37589c8c
Commit Graph

317 Commits

Author SHA1 Message Date
Xiwen Yu
b1c6f6a568 update cutlass and DeepGEMM 
Signed-off-by: Xiwen Yu <xiweny@nvidia.com>
 2025-08-26 22:43:39 -07:00
Xiwen Yu
ab7febd4d8 Merge commit '31979aefacbf80d2742c98ef30385db162788c84' into feat/b300_cu13 
Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com>
 2025-08-26 10:31:35 +08:00
Xiwen Yu
80ea0628d7 fix cubins 
Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com>
 2025-08-25 11:41:25 +08:00
dongxuy04
19a0ea363b
[TRTLLM-6743][feat] Optimize and refactor alltoall in WideEP (#6973) 
Signed-off-by: Dongxu Yang <78518666+dongxuy04@users.noreply.github.com>
Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
Signed-off-by: Dongxu Yang <dongxuy@nvidia.com>
Co-authored-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
 2025-08-24 08:15:29 -04:00
Xiwen Yu
90a9bc463d fix build error 
Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com>
 2025-08-23 23:03:32 +08:00
Xiwen Yu
808059da34 Merge remote-tracking branch 'gitlab/main' into user/xiweny/merge_main_0819 
Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com>
 2025-08-23 16:13:30 +08:00
Xiwen Yu
fa8b52ed33 fix more sm version check 
Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com>
 2025-08-23 15:17:59 +08:00
Xiwen Yu
f4de8840ec Merge remote-tracking branch 'gitlab/main' into user/xiweny/merge_main_0819 
Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com>
 2025-08-23 15:17:48 +08:00
Xiwen Yu
5391191d7f update tg cubins (temp ver) 
Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com>
 2025-08-23 15:14:38 +08:00
dominicshanshan
6f245ec78b
[None][chore] Mass integration of release/1.0 (#6864) 
Signed-off-by: Stanley Sun <190317771+StanleySun639@users.noreply.github.com>
Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>
Signed-off-by: ruodil <200874449+ruodil@users.noreply.github.com>
Signed-off-by: Yiqing Yan <yiqingy@nvidia.com>
Signed-off-by: Yanchao Lu <yanchaol@nvidia.com>
Signed-off-by: Balaram Buddharaju <169953907+brb-nv@users.noreply.github.com>
Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com>
Signed-off-by: Bo Deng <deemod@nvidia.com>
Signed-off-by: Chang Liu <9713593+chang-l@users.noreply.github.com>
Signed-off-by: Stefan Niebler <82932102+stnie@users.noreply.github.com>
Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
Signed-off-by: Superjomn <328693+Superjomn@users.noreply.github.com>
Signed-off-by: qqiao <qqiao@nvidia.com>
Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>
Signed-off-by: William Zhang <133824995+2ez4bz@users.noreply.github.com>
Signed-off-by: raayandhar <rdhar@nvidia.com>
Co-authored-by: Stanley Sun <190317771+StanleySun639@users.noreply.github.com>
Co-authored-by: ruodil <200874449+ruodil@users.noreply.github.com>
Co-authored-by: Yiqing Yan <yiqingy@nvidia.com>
Co-authored-by: Yanchao Lu <yanchaol@nvidia.com>
Co-authored-by: brb-nv <169953907+brb-nv@users.noreply.github.com>
Co-authored-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com>
Co-authored-by: Larry <197874197+LarryXFly@users.noreply.github.com>
Co-authored-by: Bo Deng <deemod@nvidia.com>
Co-authored-by: Guoming Zhang <137257613+nv-guomingz@users.noreply.github.com>
Co-authored-by: Stefan Niebler <82932102+stnie@users.noreply.github.com>
Co-authored-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
Co-authored-by: Yan Chunwei <328693+Superjomn@users.noreply.github.com>
Co-authored-by: Emma Qiao <qqiao@nvidia.com>
Co-authored-by: Yechan Kim <161688079+yechank-nvidia@users.noreply.github.com>
Co-authored-by: 2ez4bz <133824995+2ez4bz@users.noreply.github.com>
Co-authored-by: Raayan Dhar <58057652+raayandhar@users.noreply.github.com>
Co-authored-by: Zhanrui Sun <184402041+ZhanruiSunCh@users.noreply.github.com>
 2025-08-22 09:25:15 +08:00
Daniel Stokes
f7c597ec40
[None][perf] Make finalize fusion part of the tactic selection logic (#6915) 
Signed-off-by: djns99 <40156487+djns99@users.noreply.github.com>
 2025-08-21 14:08:03 -07:00
ChristinaZ
c7269ea93a
[https://nvbugs/5392414] [fix] Add customized default routing method (#6818) 
Signed-off-by: Christina Zhang <83400082+ChristinaZ@users.noreply.github.com>
 2025-08-21 16:58:41 +08:00
Dom Brown
92daec1115
[TRTLLM-7348] [feat] Enable Cross-Attention to use XQA kernels for Whisper (#7035) 
Signed-off-by: Dom Brown <3886319+DomBrown@users.noreply.github.com>
 2025-08-20 10:11:25 -04:00
zhhuang-nv
7e135d2ea7
[None][feat] Use Separate QKV Input Layout for Context MLA (#6538) 
Signed-off-by: Zhen Huang <145532724+zhhuang-nv@users.noreply.github.com>
 2025-08-19 22:04:48 +08:00
Xiwen Yu
8b532363ce Merge remote-tracking branch 'gitlab/main' into user/xiweny/merge_main_0819 
Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com>
 2025-08-19 17:02:34 +08:00
Xiwen Yu
4a95d88ce2 revert tlg kernels for ease of merge 
Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com>
 2025-08-19 11:44:36 +08:00
ChristinaZ
55f4f2d80c
[None] [fix] Fix the macro name (#6983) 
Signed-off-by: Christina Zhang <83400082+ChristinaZ@users.noreply.github.com>
 2025-08-18 03:08:32 -04:00
ChristinaZ
1e72721e8c
[None][feat] Add single block version renormalized routing kernel (#6756) 
Signed-off-by: Christina Zhang <83400082+ChristinaZ@users.noreply.github.com>
 2025-08-17 13:47:13 +08:00
bhsueh_NV
85cbd0263b
[None][feat] Support Yarn on Qwen3 (#6785) 
Signed-off-by: bhsueh <11360707+byshiue@users.noreply.github.com>
 2025-08-17 07:21:29 +08:00
Perkz Zheng
6037fe3716
[https://nvbugs/5394685][fix] proper fix for the accuracy issue in 2CTA MLA kernels (#6941) 
Signed-off-by: Perkz Zheng <67892460+PerkzZheng@users.noreply.github.com>
 2025-08-15 23:29:36 +08:00
Xiwen Yu
0bf6a18627 Fix and waive to clean L0 
Signed-off-by: Xiwen Yu <xiweny@nvidia.com>
 2025-08-15 04:37:43 -07:00
peaceh-nv
1c1d5d2495
[https://nvbugs/5451373][fix] : Fix the accuracy issue when using FP8 context MLA (#6881) 
Signed-off-by: peaceh <103117813+peaceh-nv@users.noreply.github.com>
 2025-08-15 16:53:56 +08:00
Perkz Zheng
11d89a3732
[https://nvbugs/5394685][fix] using static scheduler 2CTA MLA as WAR for an accuracy issue (#6896) 
Signed-off-by: Perkz Zheng <67892460+PerkzZheng@users.noreply.github.com>
 2025-08-15 08:51:04 +08:00
jmydurant
4200fa46d1
[None][feat] Add support for Hopper MLA chunked prefill (#6655) 
Signed-off-by: Mingyang Jiang <13463932+jmydurant@users.noreply.github.com>
 2025-08-14 10:39:26 +08:00
Perkz Zheng
58f7783ea4
[https://nvbugs/5394685][fix] the bug with spec-decoding + SWA && an accuracy issue related to 2CTA MLA (#6834) 
Signed-off-by: Perkz Zheng <67892460+PerkzZheng@users.noreply.github.com>
 2025-08-13 13:55:56 -07:00
Perkz Zheng
0fad6029f7
[TRTLLM-7093][fix] the perf regression to cvt_fp4 kernels (#6851) 
Signed-off-by: Perkz Zheng <67892460+PerkzZheng@users.noreply.github.com>
 2025-08-13 19:13:40 +08:00
Zhou Yuxin
50e5e725e9
[https://nvbugs/5412456][fix] Fix an illegal instruction was encountered (#6776) 
Signed-off-by: Zhou Yuxin <yuxinz@nvidia.com>
 2025-08-13 15:45:59 +08:00
Sergey Klevtsov
27fc35175e
[None][feat] CUTLASS MoE FC2+Finalize fusion (#3294) 
Signed-off-by: Sergey Klevtsov <sklevtsov@nvidia.com>
 2025-08-12 15:56:48 +08:00
NVJiangShao
2f2f5cc72c
[TRTLLM-6744][feat] Remove input_sf swizzle for module WideEPMoE (#6231) 
Signed-off-by: Jiang Shao <91270701+StudyingShao@users.noreply.github.com>
 2025-08-08 11:13:42 +08:00
Daniel Cámpora
efca359b66
[TRTLLM-6785][feat] BREAKING CHANGE Enable TRTLLM sampler by default (#6216) 
Signed-off-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
 2025-08-07 22:19:37 -04:00
Iman Tabrizian
82276167e6
[None][feat] Add NCCL Symmetric Integration for All Reduce (#4500) 
Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>
 2025-08-07 17:28:14 -07:00
peaceh-nv
8ec3b1de10
[None][feat] : Add FP8 context MLA support for SM120 (#6059) 
Signed-off-by: peaceh <103117813+peaceh-nv@users.noreply.github.com>
 2025-08-07 16:16:34 +08:00
hlu1
8207d5fd39
[None] [feat] Add model gpt-oss (#6645) 
Signed-off-by: Hao Lu <14827759+hlu1@users.noreply.github.com>
 2025-08-07 03:04:18 -04:00
Xiwen Yu
e27cbb57eb Ampere moe kernel should build to all arch 
Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com>
 2025-08-06 14:25:14 +08:00
Xiwen Yu
345c2bceaa update trtllm-gen sm100f cubins of gemm kernels 
Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com>
 2025-08-06 14:25:12 +08:00
Xiwen Yu
52ad4436bc disable 3xfp4 
Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com>
 2025-08-06 14:25:05 +08:00
Daniel Stokes
469a38d0d8 feat: Add support for SM103 3xFP4 tile shapes 
Signed-off-by: Daniel Stokes <dastokes@nvidia.com>
 2025-08-06 14:25:02 +08:00
Tian Zheng
3a94d80839 Update SM100f cubins 
Signed-off-by: Tian Zheng <29906817+Tom-Zheng@users.noreply.github.com>
 2025-08-06 14:25:00 +08:00
Xiwen Yu
1b846046dd fix kernel select code to recognize sm103/sm100f 
Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com>
 2025-08-06 14:24:43 +08:00
Xiwen Yu
5c09dc8304 CUDA13 breaking changes: c++ compile successful 
Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com>
 2025-08-06 14:24:40 +08:00
Haohang Huang
c9eebcb454
[TRTLLM-6674][feat] (Breaking Change) Hopper SWA non-cyclic kernels + KV reuse + Spec Dec (#6379) 
Signed-off-by: Haohang Huang <31998628+symphonylyh@users.noreply.github.com>
Signed-off-by: symphonylyh <31998628+symphonylyh@users.noreply.github.com>
 2025-08-05 07:47:41 +00:00
Perkz Zheng
03430ed379 [https://nvbugspro.nvidia.com/bug/5415268] fix illegal smem access with chunked attention (#6401) 
Signed-off-by: Perkz Zheng <67892460+PerkzZheng@users.noreply.github.com>
Co-authored-by: Sharan Chetlur <116769508+schetlur-nv@users.noreply.github.com>
 2025-08-04 11:19:58 +08:00
Jhao-Ting Chen
6edaa23c1c
[None][feat] Multi-block mode for Hopper spec dec XQA kernel (#4416) 
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
 2025-08-03 14:31:33 -07:00
yunruis
a20ab5cbdb
[https://nvbugs/5381276][fix] fix warning for fused_a_gemm (#6402) 
Signed-off-by: yunruis <205571022+yunruis@users.noreply.github.com>
 2025-08-01 09:37:21 -04:00
Yao Yao
942e080415
[fix] Fix missing fields in xqa kernel cache key (#6282) 
Signed-off-by: Yao Yao <lowsfer@users.noreply.github.com>
 2025-08-01 10:41:26 +08:00
Yukun He
93a0fd0a23
[TRTLLM-6445] feat: Enable AllReduce-associated fusion patterns in Llama3/4. (#6205) 
Signed-off-by: Yukun He <23156053+hyukn@users.noreply.github.com>
 2025-07-28 09:36:26 +08:00
Jhao-Ting Chen
54f68287fc
fix precompiled multi_query_token kernel not having is_fp8_out hash key (#6279) 
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
 2025-07-25 20:45:53 -04:00
Shiyu Li
375f74ecb2
[fix][nvbugs/5399355] Fix Lamport buffer clear issue for MNNVL TwoShot Allreduce and add FP16 support. (#6237) 
Signed-off-by: Shiyu Li <shili@nvidia.com>
 2025-07-25 08:01:40 +08:00
Perkz Zheng
706f421cb0
[Fix] the bug in the trtllm-gen heurisitcf for MLA kernels. (#6284) 
Signed-off-by: Perkz Zheng <67892460+PerkzZheng@users.noreply.github.com>
 2025-07-24 23:40:27 +08:00
Zhou Yuxin
0ffcf9a863
Update fmhaRunner.cpp to fix guardwords scan error (#6327) 
Signed-off-by: Zhou Yuxin <yuxinz@nvidia.com>
 2025-07-24 18:32:36 +08:00