kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-02-07 19:51:50 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
4,611 Commits 28 Branches 67 Tags 2 GiB
bf16fbd86c
Commit Graph

78 Commits

Author SHA1 Message Date
Yan Chunwei
ea6cd76c55
[None][refactor] simplify get_stats and get_kvcache_events with rpc (#9980) 
Signed-off-by: Yan Chunwei <328693+Superjomn@users.noreply.github.com>
Signed-off-by: Superjomn <328693+Superjomn@users.noreply.github.com>
 2025-12-22 18:23:43 +08:00
Robin Kobus
32f53910ef
[TRTLLM-909][feat] Overlap context chunks in pipeline parallel mode (#9308) 
Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
 2025-11-25 22:11:51 +01:00
QI JUN
34a6d2d28f
[TRTLLM-9302][chore] Move build config from BaseLlmArgs to TrtLlmArgs (#9249) 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
 2025-11-24 10:54:41 +08:00
Guoming Zhang
af3900a195 [https://nvbugs/5504095][fix] Unwaive test_user_specify_workspace case. (#8316) 
Signed-off-by: nv-guomingz <137257613+nv-guomingz@users.noreply.github.com>
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Mike Iovine <miovine@nvidia.com>
 2025-11-20 12:43:13 -05:00
Patrice Castonguay
d8ea0b967f
[None][fix] Moving transfer timeout test to test_llm_pytorch, fixing broken kv transfer timeout (#8892) 
Signed-off-by: Patrice Castonguay <55748270+pcastonguay@users.noreply.github.com>
 2025-11-07 07:33:51 -08:00
Anish Shanbhag
a09b38a862
[TRTLLM-8684][chore] Migrate BuildConfig to Pydantic, add a Python wrapper for KVCacheType enum (#8330) 
Signed-off-by: Anish Shanbhag <ashanbhag@nvidia.com>
 2025-10-28 09:17:26 -07:00
Anish Shanbhag
15de45d782
[TRTLLM-8682][chore] Remove auto_parallel module (#8329) 
Signed-off-by: Anish Shanbhag <ashanbhag@nvidia.com>
 2025-10-22 20:53:08 -04:00
Patrice Castonguay
879039f6d5
[https://nvbugs/5429636][feat] Kv transfer timeout (#8459) 
Signed-off-by: raayandhar <raayan.dhar@gmail.com>
Signed-off-by: Patrice Castonguay <55748270+pcastonguay@users.noreply.github.com>
Co-authored-by: raayandhar <raayan.dhar@gmail.com>
 2025-10-22 09:29:02 -04:00
Jonas Yang CN
88ea2c4ee9
[TRTLLM-7349][feat] Adding new orchestrator type -- ray (#7520) 
Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>
Co-authored-by: Yuan Tong <13075180+tongyuantongyu@users.noreply.github.com>
Co-authored-by: Erin Ho <14718778+hchings@users.noreply.github.com>
 2025-10-04 08:12:24 +08:00
Cao Dong
62010c0ab7
[None][feat] Return topk logprobs in torch backend (#7976) 
Signed-off-by: Cao Dong <87467313+dcaox@users.noreply.github.com>
 2025-09-30 09:32:37 +08:00
Yuan Tong
70c3b100eb
[#7692][fix] recognize RequestError as per-request error in background handler (#7726) 
Signed-off-by: Yuan Tong <13075180+tongyuantongyu@users.noreply.github.com>
 2025-09-24 11:11:17 +08:00
Venky
6ff0fad75e
[TRTLLM-7015] [feat] Enable prompt_logprobs in pytorch backend (#7580) 
Signed-off-by: Venky Ganesh <23023424+venkywonka@users.noreply.github.com>
 2025-09-23 18:48:10 -07:00
Yanchao Lu
70aa4e28c1
[None][ci] Test waives for the main branch 09/14 (#7698) 
Signed-off-by: Yanchao Lu <yanchaol@nvidia.com>
 2025-09-14 23:48:04 +08:00
Guoming Zhang
beefd6413e
[None][fix] fix post-merge issue raised by #5488 (#7655) 
Signed-off-by: nv-guomingz <137257613+nv-guomingz@users.noreply.github.com>
 2025-09-10 09:26:27 +08:00
Emma Qiao
d8ec546b73
[None][infra] Waive failed tests on main branch 0905 (#7564) 
Signed-off-by: qqiao <qqiao@nvidia.com>
 2025-09-05 22:46:46 +08:00
rakib-hasan
7ab8112450
[None][fix] Refactoring to avoid circular import when importing torch models (#6720) 
Signed-off-by: Rakib Hasan <rhasan@nvidia.com>
 2025-08-11 18:00:42 -04:00
amitz-nv
85af62184b
[TRTLLM-6683][feat] Support LoRA reload CPU cache evicted adapter (#6510) 
Signed-off-by: Amit Zuker <203509407+amitz-nv@users.noreply.github.com>
 2025-08-07 09:05:36 +03:00
Haohang Huang
c9eebcb454
[TRTLLM-6674][feat] (Breaking Change) Hopper SWA non-cyclic kernels + KV reuse + Spec Dec (#6379) 
Signed-off-by: Haohang Huang <31998628+symphonylyh@users.noreply.github.com>
Signed-off-by: symphonylyh <31998628+symphonylyh@users.noreply.github.com>
 2025-08-05 07:47:41 +00:00
Pengyun Lin
a15e33351d
[None][fix] Revert commit 48ddc3d & add test for disagg server with different max_num_tokens (#6259) 
Signed-off-by: Pengyun Lin <81065165+LinPoly@users.noreply.github.com>
 2025-08-04 15:09:51 +08:00
Leslie Fang
b9fe0fa7ec
[None][infra] Enable test of chunked prefill with logit post processor (#6483) 
Signed-off-by: leslie-fang25 <leslief@nvidia.com>
 2025-08-04 01:46:07 -04:00
amitz-nv
1ee7a08d2b
[5830][feat] Improve LoRA cache memory control (#6220) 
Signed-off-by: Amit Zuker <203509407+amitz-nv@users.noreply.github.com>
 2025-07-31 09:26:38 +03:00
Michal Guzek
7efe3cb0cd
[fix] Add detokenization-based stop word logic to LLM API (#5948) 
Signed-off-by: moraxu <mguzek@nvidia.com>
Signed-off-by: Michal Guzek <mguzek@nvidia.com>
 2025-07-29 10:16:59 -07:00
Pengyun Lin
48ddc3d4b9 [fix]: Revert commit 388b491 (#6143) 
Signed-off-by: Pengyun Lin <81065165+LinPoly@users.noreply.github.com>
 2025-07-22 12:48:00 +08:00
Bo Li
db77d83a2a
bug: [https://nvbugs/5368507] Fix test_generate_with_seed. (#6206) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
 2025-07-22 12:28:38 +08:00
amitz-nv
98428f330e
[TRTLLM-5826][feat] Support pytorch LoRA adapter eviction (#5616) 
Signed-off-by: Amit Zuker <203509407+amitz-nv@users.noreply.github.com>
 2025-07-20 08:00:14 +03:00
Yan Chunwei
7568deb2f1
[nvbug/5387226] chore: add propogation for trust_remote_code to AutoConfig (#6001) 
Signed-off-by: Superjomn <328693+Superjomn@users.noreply.github.com>
 2025-07-16 16:05:38 +08:00
Kaiyu Xie
aa97fbb2ad
[Nvbug/5383670] fix: switch test case to non-fp4 ckpt for more GPU coverage (#5882) 
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
 2025-07-14 20:21:46 +09:00
Pengyun Lin
3fcaa8a310 [nvbug 5327706][fix] fix mgmn postprocess error (#5835) 
Signed-off-by: Pengyun Lin <81065165+LinPoly@users.noreply.github.com>
 2025-07-14 17:17:30 +08:00
Yan Chunwei
3e1fd983c3 [nvbug5266240] chore: unwaive test_llm_with_dummy_weights (#5744) 
Signed-off-by: Superjomn <328693+Superjomn@users.noreply.github.com>
 2025-07-14 17:17:30 +08:00
Pengyun Lin
388b4919b8 [nvbug 5304752][fix] enhance _check_arguments to filter illegal requests for pytorch backend (#5541) 
Signed-off-by: Pengyun Lin <81065165+LinPoly@users.noreply.github.com>
 2025-07-14 17:17:30 +08:00
wili
2e3cf42e03
[refactor] Simplification of Speculative decoding configs (#5639) 
Signed-off-by: wili-65535 <wili-65535@users.noreply.github.com>
Co-authored-by: wili-65535 <wili-65535@users.noreply.github.com>
 2025-07-10 11:37:30 -04:00
Yiqing Yan
3aa53ec36c
[None] - Waive L0 tests (#5915) 
Signed-off-by: Yiqing Yan <yiqingy@nvidia.com>
 2025-07-10 18:33:17 +08:00
Enwei Zhu
055c4a9fe6
[NvBug 5370718, 5371538] fix: Fix incremental detokenization (#5825) 
Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
 2025-07-10 16:30:00 +08:00
Kaiyu Xie
bb5b16fcb9
feat: Return context response immediately when stream_interval > 1 (#5836) 
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
 2025-07-09 00:19:57 +09:00
Yiqing Yan
ec0d7e64b9
[Infra] - Waive L0 test (#5837) 
Signed-off-by: Yiqing Yan <yiqingy@nvidia.com>
 2025-07-08 17:54:06 +08:00
Enwei Zhu
55f86ce7ab
[NvBug 5362426] fix: Fix prompt adapter TP2 case (#5782) 
Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
 2025-07-08 16:01:36 +09:00
Bo Li
9db2e9ee47
fix: [nvbug/5368507] Fix test_generate_with_seed CI failure. (#5772) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
 2025-07-07 14:58:32 +08:00
Shunkangz
32339d1b20
Raise shut down error for each request (#4936) 
Signed-off-by: Shunkang <182541032+Shunkangz@users.noreply.github.co>
 2025-07-04 18:58:24 +09:00
nv-guomingz
d0b3d2ac65 fix:https://nvbugs/5362398 (#5609) 
Signed-off-by: nv-guomingz <137257613+nv-guomingz@users.noreply.github.com>
 2025-07-04 13:14:13 +08:00
Omer Ullman Argov
c72856188c
[ci] small multigpu speedups (#5643) 
Signed-off-by: Omer Ullman Argov <118735753+omera-nv@users.noreply.github.com>
 2025-07-03 08:06:10 -04:00
Emma Qiao
2a5fdebf10
[Infra] - Waive failed tests for main 0702 (#5671) 
Signed-off-by: qqiao <qqiao@nvidia.com>
 2025-07-02 22:05:07 -04:00
Emma Qiao
178fc3f655 [Infra][release/0.21] - waive failed tests (#5537) 
Signed-off-by: qqiao <qqiao@nvidia.com>
 2025-07-01 20:12:55 +08:00
Yan Chunwei
ee7fcbf20e [nvbug 5273941] fix: broken cyclic reference detect (#5417) 
Signed-off-by: Superjomn <328693+Superjomn@users.noreply.github.com>
 2025-07-01 20:12:55 +08:00
Emma Qiao
65c2b93284
[Infra] - Add some timeout and unwaive a test which dev fixed (#5631) 
Signed-off-by: qqiao <qqiao@nvidia.com>
 2025-07-01 05:01:32 -04:00
QI JUN
3a2c4ca77b
chore: split _build_model method for TorchLlm and TrtLlm (#5418) 
Signed-off-by: QI JUN <22017000+QiJune@users.noreply.github.com>
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
 2025-06-26 04:32:46 +08:00
Yan Chunwei
9bd42ecf9b
[TRTLLM-5208][BREAKING CHANGE] chore: make pytorch LLM the default (#5312) 
Signed-off-by: Superjomn <328693+Superjomn@users.noreply.github.com>
 2025-06-20 03:01:10 +08:00
Enwei Zhu
babdd9ce06
test: Add json_mode_eval for guided decoding evaluation (#5179) 
Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
 2025-06-16 10:03:55 +08:00
Yan Chunwei
c84e41fd9d
fix: build_config in TorchLlmArgs and avoid arbitrary args (#4972) 
Signed-off-by: Superjomn <328693+Superjomn@users.noreply.github.com>
 2025-06-15 17:51:56 -07:00
ixlmar
e055af1bc9
chore: improve disagg test failure detection (#4738) 
Signed-off-by: ixlmar <206748156+ixlmar@users.noreply.github.com>
 2025-06-15 01:28:26 +08:00
Bo Li
1b79041f5d
fix: XQA is not enabled when history_length < kMinHistoryTokensPerBlock. (#4264) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
 2025-06-11 09:38:10 +08:00