Eran Geva
|
c9771ebb99
|
[#9198][feat] Refactor dist ops in AutoDeploy (#9301)
Signed-off-by: Eran Geva <19514940+MrGeva@users.noreply.github.com>
|
2025-12-02 02:36:32 +08:00 |
|
Chenghao Zhang
|
0a2104dce9
|
[None][feat] AutoDeploy: Use the router gemm op for nemotron MOE (#9500)
Signed-off-by: Chenghao Zhang <211069071+nvchenghaoz@users.noreply.github.com>
|
2025-12-01 10:24:31 -08:00 |
|
Venky
|
639c939a4f
|
[TRTC-1943][feat] Env vars override support in LLM API (#9104)
Signed-off-by: Venky Ganesh <23023424+venkywonka@users.noreply.github.com>
|
2025-12-01 10:04:49 -08:00 |
|
brb-nv
|
f61067cbb5
|
[None][chore] Defer exposing context parallel configs (#9552)
Signed-off-by: Balaram Buddharaju <169953907+brb-nv@users.noreply.github.com>
|
2025-12-01 09:50:02 -08:00 |
|
Stefan Niebler
|
f155812eb0
|
[TRTLLM-6756][feat] Add Beam Search to TorchSampler (#8509)
Signed-off-by: Stefan Niebler <82932102+stnie@users.noreply.github.com>
|
2025-12-01 18:48:04 +01:00 |
|
Enwei Zhu
|
90345ad3f3
|
[None][fix] Skip Allreduce init for Attention DP (#9542)
Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
|
2025-12-01 21:24:40 +08:00 |
|
alel
|
4107254c82
|
[TRTLLM-6222][feat] Several perf opt for cuteDSL nvf4 gemm (#9428)
Signed-off-by: Yuhan Li <51736452+liyuhannnnn@users.noreply.github.com>
|
2025-12-01 18:10:45 +08:00 |
|
Yukun He
|
730eb3d859
|
[None][fix] Replace hash method with unique_id for cutedsl MoE runners. (#9569)
Signed-off-by: Yukun He <23156053+hyukn@users.noreply.github.com>
|
2025-12-01 17:02:33 +08:00 |
|
Neta Zmora
|
bc25fff039
|
[#9496][fix] AutoDeploy: remove auto-tuner from nvfp4_gemm forward (#9497)
Signed-off-by: Neta Zmora <96238833+nzmora-nvidia@users.noreply.github.com>
|
2025-12-01 10:04:39 +02:00 |
|
Fanrong Li
|
d69bf9f92a
|
[None][feat] add chat template kwargs support to longbench-v2 (#9544)
Signed-off-by: Fanrong Li <23290157+lfr-0531@users.noreply.github.com>
|
2025-12-01 15:59:13 +08:00 |
|
Gaoji Liu
|
9d2df04a72
|
[None][doc] fix mtp.py typo (#9307)
Signed-off-by: liugaoji <757394026@qq.com>
|
2025-11-30 21:55:13 -08:00 |
|
Li Min
|
1797e91dfd
|
[TRTLLM-6222][feat] Extend cute_dsl_nvfp4_gemm to sm103. (#9543)
Signed-off-by: Mindy Li <11663212+limin2021@users.noreply.github.com>
|
2025-12-01 10:19:36 +08:00 |
|
Enwei Zhu
|
34e2fa5c96
|
[https://nvbugs/5690172][fix] Fix Qwen3-235B ATP accuracy issue with PDL (#9530)
Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
|
2025-12-01 09:10:21 +08:00 |
|
heyuhhh
|
6e470aab72
|
[None] [feat] Optimize the algorithm part of RocketKV (#9333)
Signed-off-by: yuhangh <58161490+heyuhhh@users.noreply.github.com>
|
2025-12-01 09:04:09 +08:00 |
|
xxi
|
c12e67bb66
|
[TRTLLM-8958][feat] and [TRTLLM-8960]: create ConfigurableMoE and support TRTLLMGenFusedMoE as backend (#9486)
|
2025-12-01 08:37:07 +08:00 |
|
brb-nv
|
b77f4ffe54
|
[TRTLLM-5971][feat] Integrate helix parallelism (#9342)
Signed-off-by: Balaram Buddharaju <169953907+brb-nv@users.noreply.github.com>
|
2025-11-29 15:17:30 -08:00 |
|
dominicshanshan
|
6345074686
|
[None][chore] Weekly mass integration of release/1.1 -- rebase (#9522)
Signed-off-by: yunruis <205571022+yunruis@users.noreply.github.com>
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Mike Iovine <miovine@nvidia.com>
Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>
Signed-off-by: qgai <qgai@nvidia.com>
Signed-off-by: Balaram Buddharaju <169953907+brb-nv@users.noreply.github.com>
Signed-off-by: Yan Chunwei <328693+Superjomn@users.noreply.github.com>
Signed-off-by: Junyi Xu <219237550+JunyiXu-nv@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
Signed-off-by: nv-guomingz <137257613+nv-guomingz@users.noreply.github.com>
Signed-off-by: Jin Li <59594262+liji-nv@users.noreply.github.com>
Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com>
Signed-off-by: Vincent Zhang <vinczhang@nvidia.com>
Signed-off-by: peaceh <103117813+peaceh-nv@users.noreply.github.com>
Signed-off-by: Michal Guzek <mguzek@nvidia.com>
Signed-off-by: Michal Guzek <moraxu@users.noreply.github.com>
Signed-off-by: Chang Liu (Enterprise Products) <9713593+chang-l@users.noreply.github.com>
Signed-off-by: leslie-fang25 <leslief@nvidia.com>
Signed-off-by: Shunkang <182541032+Shunkangz@users.noreply.github.co>
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
Co-authored-by: yunruis <205571022+yunruis@users.noreply.github.com>
Co-authored-by: sunnyqgg <159101675+sunnyqgg@users.noreply.github.com>
Co-authored-by: brb-nv <169953907+brb-nv@users.noreply.github.com>
Co-authored-by: Yan Chunwei <328693+Superjomn@users.noreply.github.com>
Co-authored-by: JunyiXu-nv <219237550+JunyiXu-nv@users.noreply.github.com>
Co-authored-by: Simeng Liu <109828133+SimengLiu-nv@users.noreply.github.com>
Co-authored-by: Guoming Zhang <137257613+nv-guomingz@users.noreply.github.com>
Co-authored-by: Jin Li <59594262+liji-nv@users.noreply.github.com>
Co-authored-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com>
Co-authored-by: Vincent Zhang <vcheungyi@163.com>
Co-authored-by: peaceh-nv <103117813+peaceh-nv@users.noreply.github.com>
Co-authored-by: Michal Guzek <moraxu@users.noreply.github.com>
Co-authored-by: Chang Liu <9713593+chang-l@users.noreply.github.com>
Co-authored-by: Leslie Fang <leslief@nvidia.com>
Co-authored-by: Shunkangz <182541032+Shunkangz@users.noreply.github.com>
Co-authored-by: Shunkang <182541032+Shunkangz@users.noreply.github.co>
Co-authored-by: QI JUN <22017000+QiJune@users.noreply.github.com>
|
2025-11-29 21:48:48 +08:00 |
|
Grzegorz Kwasniewski
|
cff54fcae3
|
[#8948][feat] Support custom sharding config (#9143)
Signed-off-by: greg-kwasniewski1 <213329731+greg-kwasniewski1@users.noreply.github.com>
|
2025-11-29 05:28:05 +08:00 |
|
binghanc
|
db5b876124
|
[None][feat] support for more accurate AR calculation (#9323)
Signed-off-by: binghanc <176802681+binghanc@users.noreply.github.com>
|
2025-11-29 00:34:21 +08:00 |
|
Matthias Jouanneaux
|
f8dd494536
|
[None][perf] Helix: improve all-to-all perf for large CP size (#9494)
Signed-off-by: Matthias Jouanneaux <mjoux@nvidia.com>
Signed-off-by: Zheyu Fu <zheyuf@NVIDIA.com>
Co-authored-by: Zheyu Fu <zheyuf@nvidia.com>
|
2025-11-28 07:24:55 -08:00 |
|
mpikulski
|
e5f39ec7cf
|
[TRTLLM-9488][feat] add 'disable_flashinfer_sampling' config option (#9454)
Signed-off-by: ixlmar <206748156+ixlmar@users.noreply.github.com>
|
2025-11-28 13:00:39 +01:00 |
|
Robin Kobus
|
5eae3650c3
|
[None][fix] Pass checkpoint_format to create_input_processor (#9521)
Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
|
2025-11-28 10:32:29 +01:00 |
|
Zhenhuan Chen
|
7c3bb8534d
|
[None][chore] Revert "[None][fix] change allreduce workspace dtype to torch.int64 t… (#9538)
Signed-off-by: Zhenhuan Chen <zhenhuanc@nvidia.com>
|
2025-11-28 16:45:23 +08:00 |
|
Yukun He
|
60c43a200a
|
[None][fix] Fix on-disk cache and revise logger/statistics for AutoTuner. (#9211)
Signed-off-by: Yukun He <23156053+hyukn@users.noreply.github.com>
|
2025-11-28 13:32:21 +08:00 |
|
Lucas Liebenwein
|
2f8bd6fb36
|
[#9150][feat] AutoDeploy Nemotron-Flash support (#9504)
Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>
|
2025-11-27 18:03:57 +01:00 |
|
Enwei Zhu
|
c2562fc800
|
[https://nvbugs/5687820][fix] Remove self.abort() in DetokenizedGenerationResult (#9449)
Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
|
2025-11-27 22:54:40 +08:00 |
|
Bo Li
|
62b771877c
|
[TRTLLM-9389][chore] Refactor AlltoallMethodType. (#9388)
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
|
2025-11-27 21:09:29 +08:00 |
|
Fanrong Li
|
2d5eadf65f
|
[None][fix] fix TP support for DeepSeek-V3.2 on hopper (#9484)
Signed-off-by: Fanrong Li <23290157+lfr-0531@users.noreply.github.com>
|
2025-11-27 21:02:25 +08:00 |
|
Zhenhuan Chen
|
e47927e847
|
[None][fix] change allreduce workspace dtype to torch.int64 to avoid overflow (#9479)
Signed-off-by: Zhenhuan Chen <zhenhuanc@nvidia.com>
|
2025-11-27 17:08:41 +08:00 |
|
xxi
|
f1ed057b4c
|
[cherry-pick][https://nvbugs/5670793][fix] Solve trtllm-serve launch_disaggregated issue (#9346)
Signed-off-by: xxi <xxi@nvidia.com>
|
2025-11-27 16:13:58 +08:00 |
|
Ziyi Xiong
|
1dd55d8507
|
[https://nvbugs/5698581][fix] Init draft tokens for CUDA graph dummy request (#9505)
Signed-off-by: ziyixiong-nv <219238287+ziyixiong-nv@users.noreply.github.com>
|
2025-11-27 13:05:37 +08:00 |
|
Jiagan Cheng
|
14762e0287
|
[None][fix] Replace PYTORCH_CUDA_ALLOC_CONF with PYTORCH_ALLOC_CONF to fix deprecation warning (#9294)
Signed-off-by: Jiagan Cheng <jiaganc@nvidia.com>
|
2025-11-27 12:22:01 +08:00 |
|
QI JUN
|
a67d94963e
|
[None][chore] update comments in llm_args.py (#9472)
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
|
2025-11-27 11:06:34 +08:00 |
|
Aurelien Chartier
|
f2f197360d
|
[#9463][feat] Add revision option to trtllm commands (#9498)
Signed-off-by: Aurelien Chartier <2567591+achartier@users.noreply.github.com>
|
2025-11-27 09:30:01 +08:00 |
|
Chenghao Zhang
|
18fbda5cdb
|
[None][feat] AutoDeploy: Add A_log fusion for Mamba layers (#9422)
Signed-off-by: Chenghao Zhang <211069071+nvchenghaoz@users.noreply.github.com>
|
2025-11-26 14:39:20 -08:00 |
|
Chenghao Zhang
|
bc7b60e016
|
[None][feat] AutoDeploy: Remove redundant copies in mamba layers (#9461)
Signed-off-by: Chenghao Zhang <211069071+nvchenghaoz@users.noreply.github.com>
Co-authored-by: Suyog Gupta <41447211+suyoggupta@users.noreply.github.com>
|
2025-11-26 14:38:33 -08:00 |
|
Aurelien Chartier
|
ef7ee6a940
|
[None][feat] Add environment variable to force spec-dec number of accepted tokens (#9371)
Signed-off-by: Aurelien Chartier <2567591+achartier@users.noreply.github.com>
|
2025-11-26 07:22:16 -08:00 |
|
Chang Liu
|
b10137fdd5
|
[None][feat] Support MLA chunked prefill for DeepSeek V3.2 model (#9376)
Signed-off-by: Chang Liu (Enterprise Products) <9713593+chang-l@users.noreply.github.com>
|
2025-11-26 16:38:25 +08:00 |
|
Enwei Zhu
|
1bf2d750a2
|
[None][chore] Upgrade CuteDSL to 4.3.0 (#9444)
Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
|
2025-11-26 14:53:09 +08:00 |
|
JunyiXu-nv
|
b7308a4000
|
[https://nvbugs/5580099][fix] Cherry pick IMA issue fix from release/1.1 (#9032)
Signed-off-by: Junyi Xu <219237550+JunyiXu-nv@users.noreply.github.com>
|
2025-11-26 13:09:06 +08:00 |
|
shuyixiong
|
d8acea1db3
|
[TRTLLM-9293][feat] Enable partial weight loading to support streaming update weights (#9224)
Signed-off-by: shuyix <219646547+shuyixiong@users.noreply.github.com>
|
2025-11-26 10:59:06 +08:00 |
|
Yiqing Yan
|
1b9edf62c9
|
[None][chore] Bump version to 1.2.0rc5 (#9455)
Signed-off-by: Yiqing Yan <yiqingy@nvidia.com>
|
2025-11-26 08:37:53 +08:00 |
|
Chuang Zhu
|
0e9c7f8c07
|
[https://nvbugs/5685143][fix] avoid cudaFree overlap with cuda graph (#9438)
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
|
2025-11-25 16:20:29 -08:00 |
|
Suyog Gupta
|
e484bec82f
|
[None][chore] AutoDeploy add multi stream moe pass to default.yaml (#9430)
Signed-off-by: Suyog Gupta <41447211+suyoggupta@users.noreply.github.com>
|
2025-11-25 14:16:13 -08:00 |
|
Robin Kobus
|
32f53910ef
|
[TRTLLM-909][feat] Overlap context chunks in pipeline parallel mode (#9308)
Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
|
2025-11-25 22:11:51 +01:00 |
|
Eran Geva
|
afc52d7b93
|
[https://nvbugs/5647400] [fix] Enlarged the AllReduce workspace size to 64MB. Added AllReduce strategy to AD config. (#9145)
Signed-off-by: Eran Geva <19514940+MrGeva@users.noreply.github.com>
|
2025-11-25 10:56:07 -08:00 |
|
mpikulski
|
899fda9e47
|
[TRTLLM-9490][feat] use FlashInfer's top_k_sampling_from_probs (#9457)
Signed-off-by: ixlmar <206748156+ixlmar@users.noreply.github.com>
|
2025-11-25 18:53:53 +01:00 |
|
mpikulski
|
c5f52ab304
|
[TRTLLM-8376][feat] top-p optimization (removes redundant softmax) (#9411)
Signed-off-by: ixlmar <206748156+ixlmar@users.noreply.github.com>
|
2025-11-25 18:46:48 +01:00 |
|
YueWeng
|
cc336c4abd
|
[TRTLLM-8160][feat] Add draft token tree runtime on CDL (#8586)
Signed-off-by: Yue Weng <25103990+yweng0828@users.noreply.github.com>
|
2025-11-25 09:40:55 -05:00 |
|
Pengyun Lin
|
fa61825c74
|
[None][feat] Support custom chat template for tool calling (#9297)
Signed-off-by: Pengyun Lin <81065165+LinPoly@users.noreply.github.com>
|
2025-11-25 22:07:04 +08:00 |
|