Chuang Zhu
|
1d2b0d3d80
|
use file lock to avoid port conflict (#5123)
|
2025-06-16 14:15:37 +08:00 |
|
Zheng Duan
|
ded694b1aa
|
feat: cache reuse support (selective cache transfer) in mla cache formatter (#4749)
Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com>
|
2025-06-04 09:56:31 +08:00 |
|
Perkz Zheng
|
a089aa3225
|
[https://nvbugspro.nvidia.com/bug/5300080] Fix the bug of setting attention_chunk_size and enable chunked-attention in the generation-phase by default (#4693)
Signed-off-by: Perkz Zheng <67892460+PerkzZheng@users.noreply.github.com>
|
2025-06-03 19:02:57 -04:00 |
|
Chuang Zhu
|
f117d6abe9
|
Fabric Memory for KV Cache Transfer (#4717)
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
|
2025-05-30 15:50:21 +08:00 |
|
pcastonguay
|
d7d455e7ea
|
[feat][TRTLLM-5018] Dis serving python runtime trt backend (#4243)
* feat: Enabling dis serving with TRT backend with Python runtime
Signed-off-by: Patrice Castonguay <55748270+pcastonguay@users.noreply.github.com>
* Fixing formatting
Signed-off-by: Patrice Castonguay <55748270+pcastonguay@users.noreply.github.com>
* Fixing disagg mtp test
Signed-off-by: Patrice Castonguay <55748270+pcastonguay@users.noreply.github.com>
---------
Signed-off-by: Patrice Castonguay <55748270+pcastonguay@users.noreply.github.com>
|
2025-05-22 22:01:06 -04:00 |
|
Chuang Zhu
|
44cfd757b2
|
Agent interface impl for NIXL (#4125)
* agentConnection
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
recv
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
agentState
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
NIXL interfaces
Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>
update cmakelists
Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>
nixl improve
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
remove cppzmq
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
fix
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
transferAgent remove register
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
work for cache Test
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
reduce sleep time
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
fix test
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
intergarte
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
nixl env
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
fix rebase error
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
cpp test
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
stash for send metaData
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
loadRemoteMD after fetchRemoteMD
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
workaround for mixed gen and context
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
test_env
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
avoid port conflict in test
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
* format
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
* use std::string
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
* typo
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
* fix transferAgentTest
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
---------
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
|
2025-05-22 09:09:41 +08:00 |
|
ixlmar
|
f7ad49bb9b
|
chore: improve log-level setting UX (#4352)
Signed-off-by: ixlmar <206748156+ixlmar@users.noreply.github.com>
|
2025-05-16 09:47:44 +01:00 |
|
Chuang Zhu
|
e2318756ed
|
cacheTransceiver buffer manager (#3798)
* cacheTransceiver buffer manager
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
* fix args
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
* cpp kvCacheManager
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
* format
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
---------
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
|
2025-04-27 11:48:15 +08:00 |
|
Chuang Zhu
|
75e13f4f88
|
chore: disable some env for disagg defaultly (#3415)
* disable some env for disagg defaultly
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
* doc
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
* remove
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
---------
Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>
|
2025-04-14 10:08:10 +08:00 |
|
Shunkangz
|
8ee840159b
|
Add updateKVCacheTransfer (#2984)
Add kv cache transfer measurement
Signed-off-by: Shunkang <182541032+Shunkangz@users.noreply.github.co>
Co-authored-by: Shunkang <182541032+Shunkangz@users.noreply.github.co>
|
2025-03-25 21:45:35 +08:00 |
|
Kaiyu Xie
|
2631f21089
|
Update (#2978)
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
|
2025-03-23 16:39:35 +08:00 |
|
Dan Blanaru
|
16d2467ea8
|
Update TensorRT-LLM (#2755)
* Update TensorRT-LLM
---------
Co-authored-by: Denis Kayshev <topenkoff@gmail.com>
Co-authored-by: akhoroshev <arthoroshev@gmail.com>
Co-authored-by: Patrick Reiter Horn <patrick.horn@gmail.com>
Update
|
2025-02-11 03:01:00 +00:00 |
|
Kaiyu Xie
|
be17881062
|
Update TensorRT-LLM (#2582)
|
2024-12-16 21:50:47 -08:00 |
|
Kaiyu Xie
|
aaacc9bd68
|
Update TensorRT-LLM (#2562)
* Update TensorRT-LLM
---------
Co-authored-by: Starrick Liu <73152103+StarrickLiu@users.noreply.github.com>
|
2024-12-11 00:31:05 -08:00 |
|
Kaiyu Xie
|
535c9cc673
|
Update TensorRT-LLM (#2460)
|
2024-11-19 18:30:34 +08:00 |
|
Kaiyu Xie
|
f14d1d433c
|
Update TensorRT-LLM (#2389)
* Update TensorRT-LLM
---------
Co-authored-by: Alessio Netti <netti.alessio@gmail.com>
|
2024-10-29 22:24:38 +08:00 |
|
Kaiyu Xie
|
78f5c2936b
|
Update TensorRT-LLM (#2184)
|
2024-09-03 12:14:23 +02:00 |
|
Kaiyu Xie
|
be9cd719f7
|
Update TensorRT-LLM (#2094)
* Update TensorRT-LLM
---------
Co-authored-by: akhoroshev <arthoroshev@gmail.com>
Co-authored-by: Fabian Joswig <fjosw@users.noreply.github.com>
Co-authored-by: Tayef Shah <tayefshah@gmail.com>
Co-authored-by: lfz941 <linfanzai941@gmail.com>
|
2024-08-07 16:44:43 +08:00 |
|
Kaiyu Xie
|
bca9a33b02
|
Update TensorRT-LLM (#2008)
* Update TensorRT-LLM
---------
Co-authored-by: Timur Abishev <abishev.timur@gmail.com>
Co-authored-by: MahmoudAshraf97 <hassouna97.ma@gmail.com>
Co-authored-by: Saeyoon Oh <saeyoon.oh@furiosa.ai>
Co-authored-by: hattizai <hattizai@gmail.com>
|
2024-07-23 23:05:09 +08:00 |
|
Kaiyu Xie
|
9dbc5b38ba
|
Update TensorRT-LLM (#1891)
* Update TensorRT-LLM
---------
Co-authored-by: Marks101 <markus.schnoes@gmx.de>
Co-authored-by: lkm2835 <lkm2835@gmail.com>
|
2024-07-04 14:37:19 +08:00 |
|
Kaiyu Xie
|
f430a4b447
|
Update TensorRT-LLM (#1688)
* Update TensorRT-LLM
---------
Co-authored-by: IbrahimAmin <ibrahimamin532@gmail.com>
Co-authored-by: Fabian Joswig <fjosw@users.noreply.github.com>
Co-authored-by: Pzzzzz <hello-cd.plus@hotmail.com>
Co-authored-by: CoderHam <hemant@cohere.com>
Co-authored-by: Konstantin Lopuhin <kostia.lopuhin@gmail.com>
|
2024-05-28 20:07:49 +08:00 |
|
Kaiyu Xie
|
89ba1b1a67
|
Update TensorRT-LLM (#1554)
|
2024-05-07 23:34:28 +08:00 |
|
Kaiyu Xie
|
66ef1df492
|
Update TensorRT-LLM (#1492)
* Update TensorRT-LLM
---------
Co-authored-by: Loki <lokravi@amazon.com>
|
2024-04-24 14:44:22 +08:00 |
|
Kaiyu Xie
|
035b99e0d0
|
Update TensorRT-LLM (#1427)
* Update TensorRT-LLM
---------
Co-authored-by: meghagarwal <16129366+megha95@users.noreply.github.com>
|
2024-04-09 17:03:34 +08:00 |
|
Kaiyu Xie
|
4bb65f216f
|
Update TensorRT-LLM (#1274)
* Update TensorRT-LLM
---------
Co-authored-by: meghagarwal <16129366+megha95@users.noreply.github.com>
Co-authored-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>
|
2024-03-12 18:15:52 +08:00 |
|
Kaiyu Xie
|
c89653021e
|
Update TensorRT-LLM (20240116) (#891)
* Update TensorRT-LLM
---------
Co-authored-by: Eddie-Wang1120 <81598289+Eddie-Wang1120@users.noreply.github.com>
Co-authored-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>
|
2024-01-16 20:03:11 +08:00 |
|
Kaiyu Xie
|
deaae40bd7
|
Update TensorRT-LLM (#787)
* Update TensorRT-LLM
---------
Co-authored-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>
|
2024-01-02 17:54:32 +08:00 |
|
Kaiyu Xie
|
a75618df24
|
Update TensorRT-LLM (#667)
* Update TensorRT-LLM
---------
Co-authored-by: 0xymoro <jerrymeng100@gmail.com>
Co-authored-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>
|
2023-12-15 22:14:51 +08:00 |
|