kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-02-17 00:04:57 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
4,993 Commits 28 Branches 68 Tags 2 GiB
a9d4927235
Commit Graph

2775 Commits

Author SHA1 Message Date
nvyocox
e52eb82780
[#11234][test] Move test_ad_export_onnx to integration examples (#11260) 
Signed-off-by: yocox <yocox@nvidia.com>
 2026-02-05 11:32:57 -05:00
mpikulski
7d235cfb23
[TRTLLM-10030][chore] promote SampleState to TypeVar + typing fixes (#11281) 
Signed-off-by: ixlmar <206748156+ixlmar@users.noreply.github.com>
 2026-02-05 16:33:22 +01:00
chenfeiz0326
eae480b713
[https://nvbugs/5820874][fix] Adjust deepgemm tuning buckets to cover larger num_tokens's scope (#11259) 
Signed-off-by: Chenfei Zhang <chenfeiz@nvidia.com>
 2026-02-05 23:12:38 +08:00
mpikulski
719e82c429
[TRTLLM-10030][perf] beam search (remove GPU sync + fix batching + refactor) (#11276) 
Signed-off-by: ixlmar <206748156+ixlmar@users.noreply.github.com>
 2026-02-05 15:33:51 +01:00
Yuewei Na
0d18b2d7a4
[None][feat] Add priority-based KV cache offload filtering support (#10751) 
Signed-off-by: Yuewei Na <yna@nvidia.com>
Signed-off-by: Yuewei Na <nv-yna@users.noreply.github.com>
Co-authored-by: Yuewei Na <nv-yna@users.noreply.github.com>
 2026-02-05 05:22:56 -05:00
Chang Su
9601b17459
[#11037][fix] Fix proto-to-SamplingParams conversion bugs and add gRPC tests (#11292) 
Signed-off-by: Chang Su <chang.s.su@oracle.com>
 2026-02-05 05:00:29 -05:00
Yao Yao
d9b936be94
[None][feat] Enhance support for complex models (#11254) 
Signed-off-by: Yao Yao <lowsfer@users.noreply.github.com>
 2026-02-05 17:28:26 +08:00
Yechan Kim
36cb5f8c93
[https://nvbugs/5747920][fix] Fix multimodal serve test (#11296) 
Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>
 2026-02-05 15:12:53 +09:00
xinhe-nv
8447a96c29
[None][chore] Add failed cases into waives.txt (#11223) 
Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
 2026-02-05 00:27:24 -05:00
Jin Li
9091a193a8
[https://nvbugs/5837275][fix] Unwaive the failing case that cannot be… (#11137) 
Signed-off-by: Jin Li <59594262+liji-nv@users.noreply.github.com>
 2026-02-05 12:52:10 +08:00
dongfengy
0bd4630cd1
[https://nvbugs/5854860][fix] Fix cutedsl argmax on sm120 (#11181) 
Signed-off-by: Dongfeng Yu <dongfengy@nvidia.com>
 2026-02-04 17:15:31 -05:00
dongfengy
ad2d1df4a9
[https://nvbugs/5849697][fix] Refine QA Test List for SM120 (#11248) 
Signed-off-by: Dongfeng Yu <dongfengy@nvidia.com>
 2026-02-04 11:59:04 -08:00
Simeng Liu
d9fd8cc951
[https://nvbugs/5674665][fix] Fix accuracy drop in VSWA with KV cache block reuse (#10875) 
Signed-off-by: SimengLiu-nv <simengl@nvidia.com>
 2026-02-04 12:46:31 -05:00
Lucas Liebenwein
925d911fc0
[#10966][feat] AutoDeploy: kv cache manager integration [2/2] (#11149) 
Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>
 2026-02-04 09:44:27 -05:00
mpikulski
f0ca62b175
[None][fix] make health_generate work with beam search (#11097) 
Signed-off-by: ixlmar <206748156+ixlmar@users.noreply.github.com>
 2026-02-04 09:46:19 +01:00
xxi
02b80bfd58
[TRTLLM-9111][feat] provide the uniform test framework to test all MoE backends (#11128) 
Signed-off-by: xxi <xxi@nvidia.com>
 2026-02-04 15:57:56 +08:00
Gal Hubara-Agam
de6931bbfd
[None][fix] Fix selective_state_update perf regression for T=1 decode path (#11194) 
Signed-off-by: Gal Hubara Agam <96368689+galagam@users.noreply.github.com>
 2026-02-04 09:01:34 +02:00
chenfeiz0326
04b7db3ab5
[TRTLLM-8263][feat] Add Disagg Perf Tests (#10912) 
Signed-off-by: Chenfei Zhang <chenfeiz@nvidia.com>
 2026-02-04 10:16:11 +08:00
Lizhi Zhou
f9c4bdf6cf
[TRTLLM-8921][feat] implement gen-first disagg_service (#11020) 
Signed-off-by: Lizhi Zhou <1432185+reasonsolo@users.noreply.github.com>
 2026-02-03 15:46:11 -05:00
mpikulski
710d6ef668
[https://nvbugs/5739981][fix] unwaive tests using opt-125M (#11100) 
Signed-off-by: ixlmar <206748156+ixlmar@users.noreply.github.com>
 2026-02-03 15:21:01 +01:00
Chenjie Luo
2532eb5adc
[None][fix] Align kv_scales with modelopt HF checkpoint (#10745) 
Signed-off-by: Chenjie Luo <108829653+cjluo-nv@users.noreply.github.com>
 2026-02-03 08:03:42 -05:00
xinhe-nv
20946554f6
[None][chore] Add failed cases into waives.txt (#11216) 
Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
Signed-off-by: Jie Li <76780849+jieli-matrix@users.noreply.github.com>
Co-authored-by: Jie Li <76780849+jieli-matrix@users.noreply.github.com>
 2026-02-03 04:15:31 -05:00
xinhe-nv
b7767f682f
[None][chore] Add failed cases into waives.txt (#11202) 
Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
Signed-off-by: Jie Li <76780849+jieli-matrix@users.noreply.github.com>
Co-authored-by: Jie Li <76780849+jieli-matrix@users.noreply.github.com>
 2026-02-03 02:26:02 -05:00
xinhe-nv
03f51bb767
[None][chore] Add failed cases into waives.txt (#11193) 
Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
Signed-off-by: Jie Li <76780849+jieli-matrix@users.noreply.github.com>
Co-authored-by: Jie Li <76780849+jieli-matrix@users.noreply.github.com>
 2026-02-03 01:46:17 -05:00
Anish Shanbhag
e308eb50f4
[TRTLLM-10803][fix] Fix mocking of HuggingFace downloads in with_mocked_hf_download (#11200) 
Signed-off-by: Anish Shanbhag <ashanbhag@nvidia.com>
 2026-02-02 21:58:15 -08:00
Taylor Yeonbok Lee
304dc6f3c0
[None][chore] Print memory usage before/after accuracy test in CI (#11155) 
Signed-off-by: Taylor Yeonbok Lee <249374542+taylor-yb-lee@users.noreply.github.com>
 2026-02-03 00:23:14 -05:00
Yiqing Yan
13420178fc
[TRTLLM-10561][infra] Fix jaraco-context and wheel vulnerability (#10901) 
Signed-off-by: Yiqing Yan <yiqingy@nvidia.com>
 2026-02-03 09:54:11 +08:00
gramnarayan
585fbb2734
[#10826][feat] AutoDeploy: Eagle One-Model [2/n]: Prefill-Only Implementation (#11073) 
Signed-off-by: Govind Ramnarayan <105831528+govind-ramnarayan@users.noreply.github.com>
 2026-02-02 09:51:10 -08:00
Yanchao Lu
cd7762a2fa
[None][test] Fix an invalid test name (#11195) 
Signed-off-by: Yanchao Lu <yanchaol@nvidia.com>
 2026-02-02 23:25:51 +08:00
Rundong Li
f1b85fea4c
[None][feat] Integrate cuda.tile RMS norm kernels (#9725) 
Signed-off-by: Rundong (David) Li <davidli@nvidia.com>
Co-authored-by: Jinman Xie <jinmanx@nvidia.com>
Co-authored-by: Alexey Bylinkin <abylinkin@nvidia.com>
Co-authored-by: Qiqi Xiao <qiqix@nvidia.com>
Co-authored-by: Biao Wang <biaow@nvidia.com>
Co-authored-by: Thomas Schmid <thschmid@nvidia.com>
 2026-02-02 19:44:27 +08:00
Ivy Zhang
fa5c3ead05 [None][test] Update test list (#10883) 
Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com>
Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>
 2026-02-02 16:26:46 +08:00
Zheyu Fu
d31482686c [https://nvbugs/5680911][fix] Remove @cache decorator to enhance CI stability for unit tests using single process mode (#10730) 
Signed-off-by: Zheyu Fu <zheyuf@NVIDIA.com>
Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>
 2026-02-02 16:26:46 +08:00
Enwei Zhu
7e5e5b90b9 [https://nvbugs/5748600][ci] Update guided decoding waive list (#10904) 
Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>
 2026-02-02 16:26:46 +08:00
Yuxian Qiu
dd0a5491ba [https://nvbugs/5701445][chore] unwaive tests. (#10913) 
Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>
 2026-02-02 16:26:46 +08:00
Yuxian Qiu
40d6f23dad [https://nvbugs/5784543][chore] unwaive test. (#10906) 
Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>
 2026-02-02 16:26:46 +08:00
Lucas Liebenwein
68a18f7a3a [https://nvbugs/5814247][fix] AutoDeploy: skip mxfp4_moe test unless on Hopper (#10729) (#10850) 
Signed-off-by: Fridah-nv <201670829+Fridah-nv@users.noreply.github.com>
Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>
Co-authored-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com>
Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>
 2026-02-02 16:26:46 +08:00
Michal Guzek
fafc22e3d4 [https://nvbugs/5691730][fix] Have LoRa bf16 ckpts work with Llama 3.3-70B-fp8 (#9808) 
Signed-off-by: Michal Guzek <mguzek@nvidia.com>
Signed-off-by: Michal Guzek <moraxu@users.noreply.github.com>
Signed-off-by: Jin Li <59594262+liji-nv@users.noreply.github.com>
Co-authored-by: Jin Li <59594262+liji-nv@users.noreply.github.com>
Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>
 2026-02-02 16:26:46 +08:00
William Zhang
bc2487bc2c [https://nvbugs/5826962][fix] Fix PD disaggregation for VLMs that use mrope (#10865) 
* Why?

Commit a6a8898 enabled EPD disaggregation for VLMs that use mrope (e.g.
qwen). However, this broke PD disaggregation for these sames models.

* What?

This commit fixes this, and adds a unit test that guards against it.

Signed-off-by: William Zhang <133824995+2ez4bz@users.noreply.github.com>
Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>
 2026-02-02 16:26:46 +08:00
Lizhi Zhou
4d282bd7c1 [https://nvbugs/5821433][fix] fix test_auto_scaling for 2 GPUs (#10866) 
Signed-off-by: Lizhi Zhou <1432185+reasonsolo@users.noreply.github.com>
Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>
 2026-02-02 16:26:46 +08:00
HuiGao-NV
8fd22ac72d [https://nvbugs/5740377][fix] Prevent out-of-bounds read (#10868) 
Signed-off-by: Hui Gao <huig@nvidia.com>
Co-authored-by: Thor Johnsen <41591019+thorjohnsen@users.noreply.github.com>
Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>
 2026-02-02 16:26:46 +08:00
JunyiXu-nv
2a5b8800e1 [https://nvbugs/5754977][fix] Use free port for serve test (#10878) 
Signed-off-by: Junyi Xu <219237550+JunyiXu-nv@users.noreply.github.com>
Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>
 2026-02-02 16:26:46 +08:00
Yi Zhang
0306c0f12c
[TRTLLM-9766][feat] Integration of the KVCacheManager V2 to TRTLLM Runtime (#10659) 
Signed-off-by: yizhang-nv <187001205+yizhang-nv@users.noreply.github.com>
 2026-02-02 14:29:02 +08:00
Emma Qiao
d3df3f6feb
[None][infra] Waive failed cases and disable a stage on 02/02 (#11177) 
Signed-off-by: qqiao <qqiao@nvidia.com>
 2026-02-02 13:28:53 +08:00
Jin Li
77afcbddae
[https://nvbugs/5823284][fix] Unwaive no repro hang issue (#11138) 
Signed-off-by: Jin Li <59594262+liji-nv@users.noreply.github.com>
 2026-02-01 23:02:27 -05:00
Liao Lanyu
fef0e4b17d
[TRTLLM-10666][chore] Refactor request fetching logic for better separation of concerns (#10988) 
Signed-off-by: Lanyu Liao <lancelly@users.noreply.github.com>
Signed-off-by: Lance Liao <108499334+lancelly@users.noreply.github.com>
Signed-off-by: Liao Lanyu <108499334+lancelly@users.noreply.github.com>
Co-authored-by: Lanyu Liao <lancelly@users.noreply.github.com>
 2026-02-02 10:36:08 +08:00
Lizhi Zhou
b00e8338ec
[https://nvbugs/5834212][fix] prevent routing ctx and gen requests to the same worker; update doc for unique disagg ID (#11095) 
Signed-off-by: Lizhi Zhou <1432185+reasonsolo@users.noreply.github.com>
 2026-02-02 09:54:33 +08:00
Emma Qiao
1c8f8bed00
[None][infra] Waive failed cases for main on 1/30 (#11142) 
Signed-off-by: qqiao <qqiao@nvidia.com>
 2026-02-01 22:38:24 +08:00
Yanchao Lu
2e757e8151
[None][ci] Waive a flaky test on A10 (#11163) 
Signed-off-by: Yanchao Lu <yanchaol@nvidia.com>
 2026-02-01 00:07:23 +08:00
shuyixiong
278ced972b
[TRTLLM-9771][feat] Allow overriding quantization configs (#11062) 
Signed-off-by: shuyixiong <219646547+shuyixiong@users.noreply.github.com>
 2026-01-31 10:48:51 -05:00
bhsueh_NV
d1e4527c06
[https://nvbugs/5804683][infra] unwaive Mistral Large3 test (#10680) 
Signed-off-by: bhsueh <11360707+byshiue@users.noreply.github.com>
 2026-01-31 17:50:34 +08:00