Zheng Duan c9e2a963e0 feat: add kv cache aware router (#3831) ... * kv cache aware router Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * add tests Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * router config Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * eviction test Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> add test Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * eviction detect in worker test Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * move worker tests to single gpu Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * reduce memory fraction Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> * fix partial block Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com> --------- Signed-off-by: Zheng Duan <200704041+zhengd-nv@users.noreply.github.com>		2025-05-12 07:23:57 -04:00
..
__init__.py	cacheTransceiver buffer manager (#3798)	2025-04-27 11:48:15 +08:00
_perf_evaluator.py	chore: Cleanup deprecated APIs from LLM-API (part 1/2) (#3732)	2025-05-07 13:20:25 +08:00
build_cache.py	Update TensorRT-LLM (#2413)	2024-11-05 16:27:06 +08:00
disagg_utils.py	feat: add kv cache aware router (#3831)	2025-05-12 07:23:57 -04:00
llm_args.py	feat: add deepseek-r1 reasoning parser to trtllm-serve (#3354)	2025-05-06 08:13:04 +08:00
llm_utils.py	fix: trtllm-bench build trt engine on slurm (#3825)	2025-04-27 22:26:23 +08:00
llm.py	[nvbug/5262268][fix] Fix trtllm-bench for llama 4 (#4104)	2025-05-08 21:27:57 -07:00
mgmn_leader_node.py	fix: trtllm-bench build trt engine on slurm (#3825)	2025-04-27 22:26:23 +08:00
mgmn_worker_node.py	Update TensorRT-LLM (#2333)	2024-10-15 15:28:40 +08:00
mpi_session.py	fix: trtllm-bench build trt engine on slurm (#3825)	2025-04-27 22:26:23 +08:00
reasoning_parser.py	feat: add deepseek-r1 reasoning parser to trtllm-serve (#3354)	2025-05-06 08:13:04 +08:00
tokenizer.py	test [TRTLLM-4477,TRTLLM-4481]: Accuracy test improvement (Part 3.5): Support GSM8K and GPQA (#3483)	2025-04-22 07:38:16 +08:00
tracer.py	Update TensorRT-LLM (#2413)	2024-11-05 16:27:06 +08:00
trtllm-llmapi-launch	fix hmac in remote mpi session (#3649)	2025-04-18 17:47:51 +08:00
utils.py	fix: trtllm-bench build trt engine on slurm (#3825)	2025-04-27 22:26:23 +08:00