kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
add5e5cd93
TensorRT-LLMs/tensorrt_llm
History
pcastonguay add5e5cd93
feat: Add option to run disaggregated serving without ctx servers,… (#3243) 
* feat: Add option to run disaggregated serving without ctx servers, to benchmark gen only

Signed-off-by: Patrice Castonguay <55748270+pcastonguay@users.noreply.github.com>

* Fixing comment in sanity check

Signed-off-by: Patrice Castonguay <55748270+pcastonguay@users.noreply.github.com>

---------

Signed-off-by: Patrice Castonguay <55748270+pcastonguay@users.noreply.github.com>
2025-04-07 21:56:03 -04:00
..
_torch feat: Add option to run disaggregated serving without ctx servers,… (#3243) 2025-04-07 21:56:03 -04:00
auto_parallel chore: remove usernames from comments (#3291) 2025-04-05 13:44:28 +08:00
bench chore: refactor the LlmArgs with Pydantic and migrate remaining pybinding configs to python (#3025) 2025-04-05 13:31:48 +08:00
commands chore: refactor the LlmArgs with Pydantic and migrate remaining pybinding configs to python (#3025) 2025-04-05 13:31:48 +08:00
evaluate test: Accuracy test improvement (Part 3.2): Move Qwen tests (NvBug 5135332) (#3219) 2025-04-02 17:29:57 +08:00
executor fix: fix the acceptance rate of pytorch workflow in trtllm-bench (#3240) 2025-04-03 15:12:24 +08:00
inputs Update TensorRT-LLM (#2873) 2025-03-11 21:13:42 +08:00
layers chore: remove usernames from comments (#3291) 2025-04-05 13:44:28 +08:00
llmapi chore: remove usernames from comments (#3291) 2025-04-05 13:44:28 +08:00
models fix: redrafter sampling (#3278) 2025-04-08 07:49:32 +08:00
plugin Update TensorRT-LLM (#2936) 2025-03-18 21:25:19 +08:00
quantization test: Accuracy test improvement (Part 3.1): Extend accuracy test suite with LLM API and initial implementation of trtllm-eval (#3167) 2025-04-01 22:20:29 +08:00
runtime chore: remove usernames from comments (#3291) 2025-04-05 13:44:28 +08:00
scaffolding feat: refactor scaffolding worker and support openai api worker (#3166) 2025-04-01 18:31:52 +08:00
serve feat: Add option to run disaggregated serving without ctx servers,… (#3243) 2025-04-07 21:56:03 -04:00
tools Update TensorRT-LLM (#2849) 2025-03-04 18:44:00 +08:00
__init__.py Update (#2978) 2025-03-23 16:39:35 +08:00
_common.py Update (#2978) 2025-03-23 16:39:35 +08:00
_ipc_utils.py Update TensorRT-LLM (#2783) 2025-02-13 18:40:22 +08:00
_utils.py Update (#2978) 2025-03-23 16:39:35 +08:00
builder.py chore: remove usernames from comments (#3291) 2025-04-05 13:44:28 +08:00
disaggregated_params.py Update TensorRT-LLM (#2936) 2025-03-18 21:25:19 +08:00
functional.py perf: Add optimizations for deepseek in min latency mode (#3093) 2025-04-02 09:05:24 +08:00
graph_rewriting.py Update TensorRT-LLM (#2755) 2025-02-11 03:01:00 +00:00
logger.py Update TensorRT-LLM (#2873) 2025-03-11 21:13:42 +08:00
lora_manager.py feat: Support PeftCacheManager in Torch (#3186) 2025-04-04 12:38:08 +08:00
mapping.py fix: fix for cp > kvHeadNum (#3002) 2025-03-26 12:39:02 +08:00
module.py Update (#2978) 2025-03-23 16:39:35 +08:00
network.py chore: remove usernames from comments (#3291) 2025-04-05 13:44:28 +08:00
parameter.py Update TensorRT-LLM (#2873) 2025-03-11 21:13:42 +08:00
profiler.py Update TensorRT-LLM (#2936) 2025-03-18 21:25:19 +08:00
prompt_adapter_manager.py Update TensorRT-LLM (#2333) 2024-10-15 15:28:40 +08:00
python_plugin.py Update TensorRT-LLM (#2755) 2025-02-11 03:01:00 +00:00
sampling_params.py v1.2 (#3082) 2025-03-26 23:31:29 +08:00
top_model_mixin.py Update TensorRT-LLM (#2053) 2024-07-30 21:25:01 +08:00
version.py chore: bump version to 0.19.0.dev2025040800 (#3171) 2025-04-02 08:21:55 +08:00