Kaiyu Xie
|
504ede707e
|
[None] [fix] Fix nsys_on argument for slurm scripts (#9995)
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
|
2025-12-14 22:41:30 -08:00 |
|
Kaiyu Xie
|
0788635d6c
|
[TRTLLM-9762] [doc] Update documents for GB300 NVL72 (#9987)
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
|
2025-12-14 19:30:28 -08:00 |
|
Balaram Buddharaju
|
6a6e41f802
|
[TRTLLM-9468][chore] Update disagg benchmarking scripts to support context parallelism (#9720)
Signed-off-by: Balaram Buddharaju <169953907+brb-nv@users.noreply.github.com>
|
2025-12-12 22:29:41 -08:00 |
|
Kaiyu Xie
|
110820bb15
|
[TRTLLM-9792] [feat] Support multiple instances on single node for slurm scripts (#9900)
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
|
2025-12-12 12:12:08 +08:00 |
|
fredricz-20070104
|
341cb1a12c
|
[None][chore] Add GB300 support since it does not support segment (#9731)
Signed-off-by: FredricZ-2007 <226039983+fredricz-20070104@users.noreply.github.com>
|
2025-12-10 18:36:55 -08:00 |
|
Kaiyu Xie
|
069b05cf3d
|
[TRTLLM-9706] [doc] Update wide EP documents (#9724)
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
|
2025-12-08 11:21:11 +08:00 |
|
Chenjie Luo
|
d252101a76
|
[OMNIML-3036][doc] Re-branding TensorRT-Model-Optimizer as Nvidia Model-Optimizer (#9679)
Signed-off-by: Chenjie Luo <chenjiel@nvidia.com>
|
2025-12-07 07:14:05 -08:00 |
|
Zhenhuan Chen
|
24004535fe
|
[None][chore] refactor disaggregated scripts to use named arguments (#9581)
Signed-off-by: Zhenhuan Chen <zhenhuanc@nvidia.com>
|
2025-12-01 17:33:47 +08:00 |
|
Enwei Zhu
|
34e2fa5c96
|
[https://nvbugs/5690172][fix] Fix Qwen3-235B ATP accuracy issue with PDL (#9530)
Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
|
2025-12-01 09:10:21 +08:00 |
|
Kaiyu Xie
|
0d3c0c2156
|
[None] [chore] Enhancements and clean up to slurm scripts (#9493)
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
|
2025-11-28 16:41:41 +08:00 |
|
Enwei Zhu
|
c2562fc800
|
[https://nvbugs/5687820][fix] Remove self.abort() in DetokenizedGenerationResult (#9449)
Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
|
2025-11-27 22:54:40 +08:00 |
|
Patrice Castonguay
|
9b0f45298f
|
[None][feat] Have ability to cancel disagg request if KV cache resource are exhausted (#9155)
Signed-off-by: Patrice Castonguay <55748270+pcastonguay@users.noreply.github.com>
|
2025-11-18 20:59:17 -05:00 |
|
Zero Zeng
|
43896af1b1
|
[None][chore] benchmark refactor (#9207)
Signed-off-by: Zero Zeng <38289304+zerollzeng@users.noreply.github.com>
|
2025-11-17 23:29:28 -08:00 |
|
Zero Zeng
|
c6cce398f5
|
[TRTLLM-9053][feat] Support accuracy test and install from wheel (#9038)
Signed-off-by: Zero Zeng <38289304+zerollzeng@users.noreply.github.com>
|
2025-11-13 23:34:47 -08:00 |
|
Timothy Gao
|
96132b4274
|
[None] [doc] Add Mixed Precision Context and Generation section to Disagg (#8769)
Signed-off-by: Timothy Gao <35588167+timothygao8710@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
|
2025-11-11 23:46:12 -08:00 |
|
fredricz-20070104
|
fdd9e4fe00
|
[TRTLLM-7251][test] Get submit eplb slots empty key work (#8945)
Signed-off-by: FredricZ-2007 <226039983+fredricz-20070104@users.noreply.github.com>
|
2025-11-05 05:21:02 -08:00 |
|
Kaiyu Xie
|
db2a42f641
|
[None][chore] Add sample yaml for wide-ep example and minor fixes (#8825)
Signed-off-by: Zero Zeng <38289304+zerollzeng@users.noreply.github.com>
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
Co-authored-by: Zero Zeng <38289304+zerollzeng@users.noreply.github.com>
|
2025-11-03 07:48:34 -08:00 |
|
Lizhi Zhou
|
24167d00eb
|
[TRTLLM-8431][doc] update public doc and example, add etcd auto-scaling tests (#8602)
Signed-off-by: Lizhi Zhou <1432185+reasonsolo@users.noreply.github.com>
|
2025-10-28 17:04:53 -07:00 |
|
Patrice Castonguay
|
879039f6d5
|
[https://nvbugs/5429636][feat] Kv transfer timeout (#8459)
Signed-off-by: raayandhar <raayan.dhar@gmail.com>
Signed-off-by: Patrice Castonguay <55748270+pcastonguay@users.noreply.github.com>
Co-authored-by: raayandhar <raayan.dhar@gmail.com>
|
2025-10-22 09:29:02 -04:00 |
|
Shi Xiaowei
|
50149ac2bd
|
[None][doc] Fix the incorrect doc figure (#8536)
Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>
|
2025-10-22 10:08:55 +08:00 |
|
Zero Zeng
|
4545700fcf
|
[None][chore] Move submit.sh to python and use yaml configuration (#8003)
Signed-off-by: Zero Zeng <38289304+zerollzeng@users.noreply.github.com>
|
2025-10-20 22:36:50 -04:00 |
|
Bo Deng
|
dd25595ae8
|
[TRTLLM-7964][infra] Set nixl to default cache transceiver backend (#7926)
Signed-off-by: Bo Deng <deemod@nvidia.com>
|
2025-10-19 19:24:43 +08:00 |
|
Xianjie Qiao
|
d145e87f6f
|
[None][chore] Update disagg benchmark configs (#8289)
Signed-off-by: Xianjie <5410381+qiaoxj07@users.noreply.github.com>
Signed-off-by: Xianjie Qiao <5410381+qiaoxj07@users.noreply.github.com>
|
2025-10-13 18:15:46 +08:00 |
|
Guoming Zhang
|
9f0f52249e
|
[None][doc] Rename TensorRT-LLM to TensorRT LLM for homepage and the … (#7850)
Signed-off-by: nv-guomingz <137257613+nv-guomingz@users.noreply.github.com>
Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>
|
2025-09-25 21:02:35 +08:00 |
|
Zero Zeng
|
16bb76c31d
|
[None][chore] Update benchmark script (#7860)
Signed-off-by: Zero Zeng <38289304+zerollzeng@users.noreply.github.com>
Co-authored-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
|
2025-09-23 03:15:42 -07:00 |
|
Kaiyu Xie
|
6eef19297f
|
[None] [chore] cherry pick changes on slurm scripts from release/1.1.0rc2 (#7750)
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
|
2025-09-16 16:07:13 +08:00 |
|
Kaiyu Xie
|
23f72c8bbd
|
[None] [feat] Use numa to bind CPU (#7304)
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
|
2025-08-28 06:27:11 -04:00 |
|
Kaiyu Xie
|
8a619be828
|
[None] [chore] Make disagg example compatible with recommended usage (#7121)
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
|
2025-08-27 23:57:46 +08:00 |
|
Raayan Dhar
|
82bd1871ea
|
[None][chore] update disagg readme and scripts for pipeline parallelism (#6875)
Signed-off-by: raayandhar <rdhar@nvidia.com>
|
2025-08-27 00:53:57 -04:00 |
|
Xianjie Qiao
|
19667304b5
|
[None] [chore] Update wide-ep genonly scripts (#6995)
Signed-off-by: Xianjie <5410381+qiaoxj07@users.noreply.github.com>
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
Co-authored-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
|
2025-08-19 07:44:07 -04:00 |
|
Shi Xiaowei
|
1095dfd03c
|
[None][fix] BREAKING CHANGE: Mismatch between docs and actual commands (#6323)
|
2025-08-14 03:48:57 -04:00 |
|
Shi Xiaowei
|
fe7dda834d
|
[TRTLLM-7030][fix] Refactor the example doc of dist-serving (#6766)
Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>
|
2025-08-13 17:39:27 +08:00 |
|
Kaiyu Xie
|
47806f09d9
|
feat: Support custom repo_dir for SLURM script (#6546)
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
Co-authored-by: xxi <xxi@nvidia.com>
|
2025-08-12 22:06:59 -04:00 |
|
Guoming Zhang
|
f7f46a5017
|
doc: remove the outdated features which marked as Experimental (#5995)
Signed-off-by: nv-guomingz <137257613+nv-guomingz@users.noreply.github.com>
|
2025-08-06 22:01:42 -04:00 |
|
Pengyun Lin
|
a15e33351d
|
[None][fix] Revert commit 48ddc3d & add test for disagg server with different max_num_tokens (#6259)
Signed-off-by: Pengyun Lin <81065165+LinPoly@users.noreply.github.com>
|
2025-08-04 15:09:51 +08:00 |
|
Kaiyu Xie
|
aee35e2dbd
|
chore: Make example SLURM scripts more parameterized (#6511)
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
|
2025-08-01 12:53:15 +08:00 |
|
nv-guomingz
|
03e38c9087
|
chore: update trtllm-serve usage doc by removing backend parameter when it use torch as backend. (#6419)
Signed-off-by: nv-guomingz <137257613+nv-guomingz@users.noreply.github.com>
|
2025-07-30 11:11:06 -04:00 |
|
Kaiyu Xie
|
e58afa510e
|
doc: Add README for wide EP (#6356)
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
|
2025-07-29 00:36:12 -04:00 |
|
Kaiyu Xie
|
f08286c679
|
doc: Refactor documents and examples of disaggregated serving and wide ep (#6054)
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
|
2025-07-23 09:20:57 +08:00 |
|
Chuang Zhu
|
44c70c88f9
|
chore:[BREAKING CHANGE] use cacheTransceiverConfig as knobs for disagg service (#5234)
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
|
2025-07-17 17:42:07 +08:00 |
|
Shi Xiaowei
|
49359574c1
|
[TRTLLM-5673] Doc: ensure the disagg doc is up to date (#5938)
|
2025-07-11 17:39:05 +09:00 |
|
nv-guomingz
|
c434147366
|
chore: update doc by replacing use_cuda_graph with cuda_graph_config (#5680)
Signed-off-by: nv-guomingz <137257613+nv-guomingz@users.noreply.github.com>
|
2025-07-04 15:39:15 +09:00 |
|
Shunkangz
|
3e75320fe8
|
Add pd dynamic scaling readme (#5540)
Signed-off-by: Shunkang <182541032+Shunkangz@users.noreply.github.com>
|
2025-07-02 02:18:51 -04:00 |
|
nv-guomingz
|
578430e64c
|
[TRTLLM-5530][BREAKING CHANGE]: enhance the llm args pytorch config part 1(cuda_graph_config) (#5014)
Signed-off-by: nv-guomingz <137257613+nv-guomingz@users.noreply.github.com>
|
2025-06-30 11:05:40 +08:00 |
|
Chuang Zhu
|
947571c311
|
Fix buffer count (#5007)
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
|
2025-06-09 14:01:13 +08:00 |
|
Shunkangz
|
c835f06371
|
Refactor the first token response in PD (#4692)
Signed-off-by: Shunkang <182541032+Shunkangz@users.noreply.github.co>
Co-authored-by: Shunkang <182541032+Shunkangz@users.noreply.github.co>
|
2025-06-04 09:11:23 +08:00 |
|
Rashid Kaleem
|
19786a7961
|
[Doc] Fix readme for disaggregated serving (#4846)
Signed-off-by: Rashid K <rkaleem@nvidia.com>
Signed-off-by: Rashid Kaleem <rkaleem@nvidia.com>
Co-authored-by: Iman Tabrizian <10105175+Tabrizian@users.noreply.github.com>
|
2025-06-03 11:45:26 -04:00 |
|
Yan Chunwei
|
5506f60037
|
chore [BREAKING CHANGE]: Flatten PyTorchConfig knobs into TorchLlmArgs (#4603)
Signed-off-by: Superjomn <328693+Superjomn@users.noreply.github.com>
|
2025-05-28 18:43:04 +08:00 |
|
amirkl94
|
fbec0c3552
|
Release 0.20 to main (#4577)
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com>
Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
Signed-off-by: Martin Marciniszyn Mehringer <11665257+MartinMarciniszyn@users.noreply.github.com>
Signed-off-by: Yuan Tong <13075180+tongyuantongyu@users.noreply.github.com>
Signed-off-by: Yukun He <23156053+hyukn@users.noreply.github.com>
Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
Signed-off-by: Venky <23023424+venkywonka@users.noreply.github.com>
Signed-off-by: Ruodi <200874449+ruodil@users.noreply.github.com>
Signed-off-by: Stefan Niebler <82932102+stnie@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
Signed-off-by: Faraz Khoubsirat <58580514+farazkh80@users.noreply.github.com>
Signed-off-by: moraxu <mguzek@nvidia.com>
Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>
Signed-off-by: Jinyang Yuan <154768711+jinyangyuan-nvidia@users.noreply.github.com>
Co-authored-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
Co-authored-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com>
Co-authored-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
Co-authored-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: Martin Marciniszyn Mehringer <11665257+MartinMarciniszyn@users.noreply.github.com>
Co-authored-by: Yuan Tong <13075180+tongyuantongyu@users.noreply.github.com>
Co-authored-by: Yukun He <23156053+hyukn@users.noreply.github.com>
Co-authored-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
Co-authored-by: Venky <23023424+venkywonka@users.noreply.github.com>
Co-authored-by: ruodil <200874449+ruodil@users.noreply.github.com>
Co-authored-by: stnie <82932102+stnie@users.noreply.github.com>
Co-authored-by: Simeng Liu <109828133+SimengLiu-nv@users.noreply.github.com>
Co-authored-by: Faraz <58580514+farazkh80@users.noreply.github.com>
Co-authored-by: Michal Guzek <moraxu@users.noreply.github.com>
Co-authored-by: Iman Tabrizian <10105175+Tabrizian@users.noreply.github.com>
Co-authored-by: Jinyang Yuan <154768711+jinyangyuan-nvidia@users.noreply.github.com>
|
2025-05-28 16:25:33 +08:00 |
|
pcastonguay
|
d7d455e7ea
|
[feat][TRTLLM-5018] Dis serving python runtime trt backend (#4243)
* feat: Enabling dis serving with TRT backend with Python runtime
Signed-off-by: Patrice Castonguay <55748270+pcastonguay@users.noreply.github.com>
* Fixing formatting
Signed-off-by: Patrice Castonguay <55748270+pcastonguay@users.noreply.github.com>
* Fixing disagg mtp test
Signed-off-by: Patrice Castonguay <55748270+pcastonguay@users.noreply.github.com>
---------
Signed-off-by: Patrice Castonguay <55748270+pcastonguay@users.noreply.github.com>
|
2025-05-22 22:01:06 -04:00 |
|