kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
d4d21a106e
TensorRT-LLMs/tensorrt_llm/_torch/pyexecutor
History
Iman Tabrizian d4d21a106e
[fix] Release slots with spec decode + disagg (#5975) (#6032) 
Signed-off-by: Iman Tabrizian <itabrizian@nvidia.com>
Signed-off-by: Iman Tabrizian <10105175+Tabrizian@users.noreply.github.com>
Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>
2025-07-17 12:58:18 +08:00
..
__init__.py Update TensorRT-LLM (#2755) 2025-02-11 03:01:00 +00:00
_util.py [TRTLLM-5493] Add core infrastructure to enable loading of custom checkpoint formats (#5372) 2025-07-17 00:50:30 +08:00
config_utils.py feat: support kv cache reuse for MLA (#3571) 2025-05-15 15:22:21 +08:00
config.py [TRTLLM-5493] Add core infrastructure to enable loading of custom checkpoint formats (#5372) 2025-07-17 00:50:30 +08:00
cuda_graph_runner.py [refactor] Simplification of Speculative decoding configs (#5639) 2025-07-10 11:37:30 -04:00
finish_reason.py [TRTLLM-5974][feat] Support disaggregated serving in TRTLLM Sampler (#5328) 2025-06-25 17:41:36 +02:00
grammar_matcher.py feat: TRTLLM-5941 Upgrade xgrammar to 0.1.18 (#5364) 2025-07-01 20:12:55 +08:00
guided_decoder.py Reintroduce with perf fixes: feature: unify new_tokens format sample state to trtllm samper tokens format (#5513) 2025-06-30 11:58:59 -07:00
handle_logits.py feat: Optimize TRTLLM Sampler perf single beam single step (#5550) 2025-07-07 15:44:47 +02:00
kv_cache_transceiver.py Solve underallocation in VSWA+/VGQA (#4667) 2025-06-12 12:12:46 +08:00
layerwise_nvtx_marker.py Update TensorRT-LLM (#2849) 2025-03-04 18:44:00 +08:00
llm_request.py feat: add MultimodalParams & putting all multimodal params into it and refactor HyperCLOVAX & Qwen2/2.5-VL (#5522) 2025-07-07 18:03:12 -07:00
make_decoding_batch_input_output.py feat: Optimize TRTLLM Sampler perf single beam single step (#5550) 2025-07-07 15:44:47 +02:00
model_engine.py [TRTLLM-5493] Add core infrastructure to enable loading of custom checkpoint formats (#5372) 2025-07-17 00:50:30 +08:00
py_executor_creator.py [refactor] Clean up drafter/resource manager creation logic (#5805) 2025-07-16 12:45:46 -07:00
py_executor.py [fix] Release slots with spec decode + disagg (#5975) (#6032) 2025-07-17 12:58:18 +08:00
resource_manager.py Fix: Enhance ModelConfig for kv cache size calculations (#5868) 2025-07-16 14:41:31 -07:00
sampler.py [nvbugs/5345391] fix: chunked prefill + overlap scheduling (#5761) 2025-07-14 17:17:30 +08:00
scheduler.py Reintroduce with perf fixes: feature: unify new_tokens format sample state to trtllm samper tokens format (#5513) 2025-06-30 11:58:59 -07:00
seq_slot_manager.py feat: Optimize TRTLLM Sampler perf single beam single step (#5550) 2025-07-07 15:44:47 +02:00