Zheng Duan c9e2a963e0 feat: add kv cache aware router (#3831) ... * kv cache aware router Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * add tests Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * router config Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * eviction test Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> add test Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * eviction detect in worker test Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * move worker tests to single gpu Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * reduce memory fraction Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * fix partial block Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> --------- Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com>		2025-05-12 07:23:57 -04:00
..
_torch	[fix] Fix relaxed acceptance to support enabling it in context phase (#4126)	2025-05-09 14:11:14 +08:00
api_stability	feat: Support the Structural Tag in guided decoding (#4066)	2025-05-12 17:24:50 +08:00
bindings	chore: Cleanup deprecated APIs from LLM-API (part 1/2) (#3732)	2025-05-07 13:20:25 +08:00
disaggregated	feat: add kv cache aware router (#3831)	2025-05-12 07:23:57 -04:00
llmapi	feat: Support the Structural Tag in guided decoding (#4066)	2025-05-12 17:24:50 +08:00
others	test: reorganize tests folder hierarchy (#2996)	2025-03-27 12:07:53 +08:00
scaffolding	[TRTLLM-4911] feat(scaffolding): make sampling_params only setable by controller (#4151)	2025-05-12 15:29:09 +08:00
tools	Update TensorRT-LLM (#2936)	2025-03-18 21:25:19 +08:00
trt	Unify two versions of AllReduce custom op (#3032)	2025-04-22 21:58:42 +08:00
utils	refactor: Move ModelSpec to core library (#3980)	2025-05-04 01:39:09 +08:00
conftest.py	Add thread leak check and fix thread/memory leak issues. (#3270)	2025-04-08 19:03:18 +08:00
dump_checkpoint_stats.py	Update TensorRT-LLM (#2936)	2025-03-18 21:25:19 +08:00
profile_utils.py	Update TensorRT-LLM (#2936)	2025-03-18 21:25:19 +08:00
pytest.ini	move the reset models into examples/models/core directory (#3555)	2025-04-19 20:48:59 -07:00
test_model_runner_cpp.py	Update TensorRT-LLM (#2936)	2025-03-18 21:25:19 +08:00
test_pip_install.py	relax the limitation of setuptools (#2992)	2025-03-24 13:36:10 +08:00