kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-13 22:18:36 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
4,081 Commits 25 Branches 59 Tags 2.1 GiB
36c9e7cfe6
Commit Graph

359 Commits

Author SHA1 Message Date
Frank
f6df9eb2a6
[TRTLLM-9089][chore] Port prepare_dataset into trtllm-bench (#9250) 2025-12-08 10:37:40 -08:00
Kaiyu Xie
069b05cf3d
[TRTLLM-9706] [doc] Update wide EP documents (#9724) 
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
 2025-12-08 11:21:11 +08:00
Chenjie Luo
d252101a76
[OMNIML-3036][doc] Re-branding TensorRT-Model-Optimizer as Nvidia Model-Optimizer (#9679) 
Signed-off-by: Chenjie Luo <chenjiel@nvidia.com>
 2025-12-07 07:14:05 -08:00
QI JUN
d4f68195c3 [TRTLLM-9092][doc] link to modelopt checkpoints in quick start guide (#9571) 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Mike Iovine <miovine@nvidia.com>
 2025-12-05 17:50:12 -05:00
QI JUN
0406949f32 [TRTLLM-9093][doc] update hyper links in overview (#9568) 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Mike Iovine <miovine@nvidia.com>
 2025-12-05 17:50:12 -05:00
Yiqing Yan
6ebdf1c304 [None][infra] Updated Linux installation guide (#9485) 
Signed-off-by: Yiqing Yan <yiqingy@nvidia.com>
Co-authored-by: Yanchao Lu <yanchaol@nvidia.com>
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Mike Iovine <miovine@nvidia.com>
 2025-12-05 17:50:12 -05:00
Enwei Zhu
b46e78e263 [TRTLLM-9157][doc] Guided decoding doc improvement (#9359) 
Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Mike Iovine <miovine@nvidia.com>
 2025-12-05 17:50:12 -05:00
QI JUN
0915c4e3a1 [TRTLLM-9086][doc] Clean up TODOs in documentation (#9292) 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Mike Iovine <miovine@nvidia.com>
 2025-12-05 17:50:12 -05:00
Pengyun Lin
c6dc68a28e [None][doc] VDR 1.0 trtllm-serve doc enhancement (#9443) 
Signed-off-by: Pengyun Lin <81065165+LinPoly@users.noreply.github.com>
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Mike Iovine <miovine@nvidia.com>
 2025-12-05 17:50:12 -05:00
jthomson04
6332bf27e6 [TRTLLM-9199][docs] KV Connector Docs (#9325) 
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Mike Iovine <miovine@nvidia.com>
 2025-12-05 17:50:12 -05:00
Robin Kobus
faf682b8bc
[TRTLLM-7136][feat] Update load_weights method to include mapping parameter in checkpoint loaders (#9583) 
Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
 2025-12-05 16:07:20 +01:00
Kaiyu Xie
cb87c44912
[TRTLLM-9562] [doc] Add Deployment Guide for Kimi K2 Thinking on TensorRT LLM - Blackwell (#9711) 
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
 2025-12-04 19:20:06 -08:00
Thor Johnsen
33224560b8
[None][doc] Added line about partial reuse (#7846) 
Signed-off-by: thorjohnsen <41591019+thorjohnsen@users.noreply.github.com>
 2025-12-04 18:19:32 -08:00
brb-nv
5d6edc3944
[None][doc] Add feature docs for helix parallelism (#9684) 
Signed-off-by: Balaram Buddharaju <169953907+brb-nv@users.noreply.github.com>
 2025-12-04 18:08:40 -08:00
QI JUN
d11acee22d
[TRTLLM-9085][doc] fix math formula rendering issues in github (#9605) 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
 2025-12-02 10:18:16 +08:00
dominicshanshan
6345074686
[None][chore] Weekly mass integration of release/1.1 -- rebase (#9522) 
Signed-off-by: yunruis <205571022+yunruis@users.noreply.github.com>
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Mike Iovine <miovine@nvidia.com>
Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>
Signed-off-by: qgai <qgai@nvidia.com>
Signed-off-by: Balaram Buddharaju <169953907+brb-nv@users.noreply.github.com>
Signed-off-by: Yan Chunwei <328693+Superjomn@users.noreply.github.com>
Signed-off-by: Junyi Xu <219237550+JunyiXu-nv@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
Signed-off-by: nv-guomingz <137257613+nv-guomingz@users.noreply.github.com>
Signed-off-by: Jin Li <59594262+liji-nv@users.noreply.github.com>
Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com>
Signed-off-by: Vincent Zhang <vinczhang@nvidia.com>
Signed-off-by: peaceh <103117813+peaceh-nv@users.noreply.github.com>
Signed-off-by: Michal Guzek <mguzek@nvidia.com>
Signed-off-by: Michal Guzek <moraxu@users.noreply.github.com>
Signed-off-by: Chang Liu (Enterprise Products) <9713593+chang-l@users.noreply.github.com>
Signed-off-by: leslie-fang25 <leslief@nvidia.com>
Signed-off-by: Shunkang <182541032+Shunkangz@users.noreply.github.co>
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
Co-authored-by: yunruis <205571022+yunruis@users.noreply.github.com>
Co-authored-by: sunnyqgg <159101675+sunnyqgg@users.noreply.github.com>
Co-authored-by: brb-nv <169953907+brb-nv@users.noreply.github.com>
Co-authored-by: Yan Chunwei <328693+Superjomn@users.noreply.github.com>
Co-authored-by: JunyiXu-nv <219237550+JunyiXu-nv@users.noreply.github.com>
Co-authored-by: Simeng Liu <109828133+SimengLiu-nv@users.noreply.github.com>
Co-authored-by: Guoming Zhang <137257613+nv-guomingz@users.noreply.github.com>
Co-authored-by: Jin Li <59594262+liji-nv@users.noreply.github.com>
Co-authored-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com>
Co-authored-by: Vincent Zhang <vcheungyi@163.com>
Co-authored-by: peaceh-nv <103117813+peaceh-nv@users.noreply.github.com>
Co-authored-by: Michal Guzek <moraxu@users.noreply.github.com>
Co-authored-by: Chang Liu <9713593+chang-l@users.noreply.github.com>
Co-authored-by: Leslie Fang <leslief@nvidia.com>
Co-authored-by: Shunkangz <182541032+Shunkangz@users.noreply.github.com>
Co-authored-by: Shunkang <182541032+Shunkangz@users.noreply.github.co>
Co-authored-by: QI JUN <22017000+QiJune@users.noreply.github.com>
 2025-11-29 21:48:48 +08:00
Grzegorz Kwasniewski
cff54fcae3
[#8948][feat] Support custom sharding config (#9143) 
Signed-off-by: greg-kwasniewski1 <213329731+greg-kwasniewski1@users.noreply.github.com>
 2025-11-29 05:28:05 +08:00
Lucas Liebenwein
2f8bd6fb36
[#9150][feat] AutoDeploy Nemotron-Flash support (#9504) 
Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>
 2025-11-27 18:03:57 +01:00
Liao Lanyu
5425d96757
[TRTLLM-9513][docs] Qwen3 deployment guide (#9488) 
Signed-off-by: Lanyu Liao <laliao@laliao-mlt.client.nvidia.com>
Co-authored-by: Lanyu Liao <laliao@laliao-mlt.client.nvidia.com>
 2025-11-27 14:12:35 +08:00
Jiagan Cheng
14762e0287
[None][fix] Replace PYTORCH_CUDA_ALLOC_CONF with PYTORCH_ALLOC_CONF to fix deprecation warning (#9294) 
Signed-off-by: Jiagan Cheng <jiaganc@nvidia.com>
 2025-11-27 12:22:01 +08:00
QI JUN
c6fa042332
[TRTLLM-9085][doc] fix math formula rendering issues (#9481) 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
 2025-11-27 10:09:12 +08:00
Zhenhuan Chen
943b05e2d3
[TRTLLM-9179][feat] add pp_partition to customize each rank's layer number (#9003) 
Signed-off-by: Zhenhuan Chen <zhenhuanc@nvidia.com>
 2025-11-13 10:34:17 +08:00
Yiqing Yan
78fac1f665
[None][chore] Lock onnx version <1.20.0 and remove WAR for TRT 10.13 (#9006) 
Signed-off-by: Yiqing Yan <yiqingy@nvidia.com>
Signed-off-by: Yanchao Lu <yanchaol@nvidia.com>
Co-authored-by: Yanchao Lu <yanchaol@nvidia.com>
 2025-11-10 10:34:06 +08:00
mpikulski
1944fb15af
[None][fix] add missing CLI option in multimodal example (#8977) 
Signed-off-by: ixlmar <206748156+ixlmar@users.noreply.github.com>
 2025-11-07 09:06:08 +01:00
Zhanrui Sun
4de31bece2
[TRTLLM-8994][infra] upgrade to DLFW 25.10 and pytorch 2.9.0 / triton 3.5.0 (#8838) 
Signed-off-by: ZhanruiSunCh <184402041+ZhanruiSunCh@users.noreply.github.com>
Signed-off-by: Yanchao Lu <yanchaol@nvidia.com>
Co-authored-by: Yanchao Lu <yanchaol@nvidia.com>
 2025-11-04 18:59:34 +08:00
Anish Shanbhag
6a6317727b
[TRTLLM-8680][doc] Add table with one-line deployment commands to docs (#8173) 
Signed-off-by: Anish Shanbhag <ashanbhag@nvidia.com>
 2025-11-03 17:42:41 -08:00
Guoming Zhang
65b793c77e
[None][doc] Add the missing content for model support section and fix valid links for long_sequence.md (#8869) 
Signed-off-by: nv-guomingz <137257613+nv-guomingz@users.noreply.github.com>
 2025-11-03 02:06:04 -08:00
Yan Chunwei
271a981f1f
[None][doc] Add LLM-API API change principle (#8350) 
Signed-off-by: Yan Chunwei <328693+Superjomn@users.noreply.github.com>
 2025-11-03 01:47:15 -08:00
dongfengy
6424f7e55f
[None][doc] Clarify the perf best practice and supported hardware for gptoss (#8665) 
Signed-off-by: Dongfeng Yu <dongfengy@nvidia.com>
Signed-off-by: dongfengy <99041270+dongfengy@users.noreply.github.com>
 2025-10-31 10:11:59 -07:00
Yi Zhang
496b419791
[None][doc] Add doc for torch.compile & piecewise cuda graph (#8527) 
Signed-off-by: yizhang-nv <187001205+yizhang-nv@users.noreply.github.com>
 2025-10-29 21:15:46 -07:00
Sharan Chetlur
a2e964d9a8
[None][doc] Minor doc update to disagg-serving (#8768) 
Signed-off-by: Sharan Chetlur <116769508+schetlur-nv@users.noreply.github.com>
 2025-10-29 17:38:06 -07:00
Robin Kobus
990b0c0c47
[TRTLLM-7159][docs] Add documentation for additional outputs (#8325) 
Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
 2025-10-27 09:52:04 +01:00
Zheng Duan
e666a704f5
[None][doc] add visualization of perf metrics in time breakdown tool doc (#8530) 
Signed-off-by: zhengd-nv <200704041+zhengd-nv@users.noreply.github.com>
 2025-10-23 22:09:21 -04:00
yunruis
8c9fda4b85
[None][doc] Paragraph adjustment and fix statistic (#8568) 
Signed-off-by: yunruis <205571022+yunruis@users.noreply.github.com>
 2025-10-22 03:26:09 -04:00
Shi Xiaowei
50149ac2bd
[None][doc] Fix the incorrect doc figure (#8536) 
Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>
 2025-10-22 10:08:55 +08:00
Shi Xiaowei
a0024f4d34
[None][doc] Facilitates the integration of the transfer agent (#7867) 
Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>
 2025-10-21 20:06:24 +08:00
Yueh-Ting (eop) Chen
85088dce05
[None][chore] Update feature combination matrix for SWA kv cache reuse (#8529) 
Signed-off-by: eopXD <yuehtingc@nvidia.com>
 2025-10-21 04:41:44 -04:00
Yechan Kim
85d5aa7763
[None][feat] Support kv_cahce_reuse for HyperCLOVAX-Vision model (#7789) 
Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>
 2025-10-21 11:11:24 +09:00
Bo Deng
dd25595ae8
[TRTLLM-7964][infra] Set nixl to default cache transceiver backend (#7926) 
Signed-off-by: Bo Deng <deemod@nvidia.com>
 2025-10-19 19:24:43 +08:00
h-guo18
55fed1873c
[None][chore] AutoDeploy: cleanup old inference optimizer configs (#8039) 
Signed-off-by: h-guo18 <67671475+h-guo18@users.noreply.github.com>
Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>
Co-authored-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>
 2025-10-17 15:55:57 -04:00
Leslie Fang
023e515d33
[None][chore] Combine two documents of feature combination matrix (#8442) 
Signed-off-by: leslie-fang25 <leslief@nvidia.com>
 2025-10-17 14:31:33 +08:00
xiweny
89d03d7668
[https://nvbugs/5532789] [doc] Add documents about CUDA 12.9 (#8411) 
Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com>
 2025-10-16 00:05:17 -07:00
Erin
f4e7738f65
[None][doc] Ray orchestrator initial doc (#8373) 
Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>
 2025-10-14 21:17:57 -07:00
Kaiyu Xie
c822c117ce
[None] [docs] Update TPOT/ITL docs (#8378) 
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
 2025-10-14 20:50:54 -07:00
Kaiyu Xie
040103ab56
[None] [blog] Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) (#8323) 
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
 2025-10-13 06:37:17 -07:00
Guoming Zhang
989c25fcba
[None][doc] Add qwen3-next doc into deployment guid and test case into L0. (#8288) 
Signed-off-by: nv-guomingz <137257613+nv-guomingz@users.noreply.github.com>
Co-authored-by: Faradawn Yang <faradawny@gmail.com>
Co-authored-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
 2025-10-13 10:25:45 +08:00
Guoming Zhang
656d73087e
[None][doc] Fix several invalid ref links in deployment guide sections. (#8287) 
Signed-off-by: nv-guomingz <137257613+nv-guomingz@users.noreply.github.com>
 2025-10-13 10:22:32 +08:00
Guoming Zhang
a193867f8f
[None][doc] Refine deployment guide by renaming TRT-LLM to TensorRT L… (#8214) 
Signed-off-by: nv-guomingz <137257613+nv-guomingz@users.noreply.github.com>
 2025-10-09 17:11:24 +08:00
Jonas Yang CN
88ea2c4ee9
[TRTLLM-7349][feat] Adding new orchestrator type -- ray (#7520) 
Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>
Co-authored-by: Yuan Tong <13075180+tongyuantongyu@users.noreply.github.com>
Co-authored-by: Erin Ho <14718778+hchings@users.noreply.github.com>
 2025-10-04 08:12:24 +08:00
WeiHaocheng
563e588e56
[None][doc] Scaffolding tech blog fix a typo (#8042) 
Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
 2025-09-28 10:29:01 -04:00