Iman Tabrizian af04b6f6aa bug: Fix hang bug when context server doesn't have enough capacity for KV Cache (#3095) ... * Fix hang bug when KV cache is low Signed-off-by: Iman Tabrizian <itabrizian@nvidia.com> * Review comments Signed-off-by: Iman Tabrizian <itabrizian@nvidia.com> * Fix attentiondp typo Signed-off-by: Iman Tabrizian <itabrizian@nvidia.com> * Add CI test for this case Signed-off-by: Iman Tabrizian <itabrizian@nvidia.com> * fix: Fix the insertion order for responder futures Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com> * fix: Fix disagg CPP Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com> --------- Signed-off-by: Iman Tabrizian <itabrizian@nvidia.com> Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>		2025-04-21 15:16:55 +08:00
..
_torch	bug: Fix hang bug when context server doesn't have enough capacity for KV Cache (#3095)	2025-04-21 15:16:55 +08:00
auto_parallel	chore: remove usernames from comments (#3291)	2025-04-05 13:44:28 +08:00
bench	feat: adding multimodal (only image for now) support in trtllm-bench (#3490)	2025-04-18 07:06:16 +08:00
commands	feat: Disaggregated router class (#3584)	2025-04-19 00:34:12 +08:00
evaluate	feat: adding multimodal (only image for now) support in trtllm-bench (#3490)	2025-04-18 07:06:16 +08:00
executor	fix hmac in remote mpi session (#3649)	2025-04-18 17:47:51 +08:00
inputs	feat: adding multimodal (only image for now) support in trtllm-bench (#3490)	2025-04-18 07:06:16 +08:00
layers	feat: Add FP8 support for SM 120 (#3248)	2025-04-14 16:05:41 -07:00
llmapi	feat: Disaggregated router class (#3584)	2025-04-19 00:34:12 +08:00
models	feat: [Deepseek] Add trtllm-gen MOE FP4 MOE backend (#3387)	2025-04-21 10:01:33 +08:00
plugin	Update TensorRT-LLM (#2936)	2025-03-18 21:25:19 +08:00
quantization	feat: [Deepseek] Add trtllm-gen MOE FP4 MOE backend (#3387)	2025-04-21 10:01:33 +08:00
runtime	feat: Offloading Multimodal embedding table to CPU in Chunked Prefill Mode (#3380)	2025-04-21 14:31:01 +08:00
scaffolding	feat: Make scaffolding Controller more generic #3408 (#3416)	2025-04-12 21:35:38 +08:00
serve	feat: trtllm-serve multimodal support (#3590)	2025-04-19 05:01:28 +08:00
tools	test: Fix breaking Phi3 multimodal tests (#3544)	2025-04-15 08:02:34 +08:00
__init__.py	Update (#2978)	2025-03-23 16:39:35 +08:00
_common.py	Update (#2978)	2025-03-23 16:39:35 +08:00
_ipc_utils.py	fix: Proper error bubbling for PyExecutor (#3321)	2025-04-15 14:49:46 +08:00
_utils.py	test: add kv cache event tests for disagg workers (#3602)	2025-04-18 18:30:19 +08:00
builder.py	chore: remove usernames from comments (#3291)	2025-04-05 13:44:28 +08:00
disaggregated_params.py	Update TensorRT-LLM (#2936)	2025-03-18 21:25:19 +08:00
functional.py	Cache sin cos in model instead of global LRU cache. (#3378)	2025-04-14 11:19:09 +08:00
graph_rewriting.py	Update TensorRT-LLM (#2755)	2025-02-11 03:01:00 +00:00
logger.py	Update TensorRT-LLM (#2873)	2025-03-11 21:13:42 +08:00
lora_manager.py	Feat/ Integrate peftCacheManager in PyExecutor creation (#3372)	2025-04-15 15:14:43 +08:00
mapping.py	chore: unify pp_layers helpers (#3429)	2025-04-15 04:49:17 +08:00
module.py	Update (#2978)	2025-03-23 16:39:35 +08:00
network.py	chore: remove usernames from comments (#3291)	2025-04-05 13:44:28 +08:00
parameter.py	Update TensorRT-LLM (#2873)	2025-03-11 21:13:42 +08:00
profiler.py	Update TensorRT-LLM (#2936)	2025-03-18 21:25:19 +08:00
prompt_adapter_manager.py	Update TensorRT-LLM (#2333)	2024-10-15 15:28:40 +08:00
python_plugin.py	Update TensorRT-LLM (#2755)	2025-02-11 03:01:00 +00:00
sampling_params.py	v1.2 (#3082)	2025-03-26 23:31:29 +08:00
top_model_mixin.py	Update TensorRT-LLM (#2053)	2024-07-30 21:25:01 +08:00
version.py	chore: bump version to 0.20.0rc0 (#3561)	2025-04-16 11:41:21 +08:00