Zheng Duan c9e2a963e0 feat: add kv cache aware router (#3831) ... * kv cache aware router Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * add tests Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * router config Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * eviction test Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> add test Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * eviction detect in worker test Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * move worker tests to single gpu Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * reduce memory fraction Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * fix partial block Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> --------- Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com>		2025-05-12 07:23:57 -04:00
..
__init__.py	Update TensorRT-LLM (#613)	2023-12-08 17:49:24 +08:00
bench.py	Update TensorRT-LLM (#2849)	2025-03-04 18:44:00 +08:00
build.py	Update (#2978)	2025-03-23 16:39:35 +08:00
eval.py	test [TRTLLM-4477,TRTLLM-4481]: Accuracy test improvement (Part 3.5): Support GSM8K and GPQA (#3483)	2025-04-22 07:38:16 +08:00
prune.py	Update TensorRT-LLM (#2008)	2024-07-23 23:05:09 +08:00
refit.py	Update TensorRT-LLM (#2532)	2024-12-04 21:16:56 +08:00
serve.py	feat: add kv cache aware router (#3831)	2025-05-12 07:23:57 -04:00