TensorRT-LLMs/docs/source/torch/features/feature_combination_matrix.md
Leslie Fang d980928c96
[doc] update the doc of feature combination matrix (#6441)
Signed-off-by: leslie-fang25 <leslief@nvidia.com>
2025-07-30 18:48:49 +08:00

5.1 KiB

Feature Combination Matrix

Feature Overlap Scheduler CUDA Graph Attention Data Parallelism Disaggregated Serving Chunked Prefill MTP EAGLE-3(One Model Engine) EAGLE-3(Two Model Engine) Torch Sampler TLLM C++ Sampler KV Cache Reuse Slide Window Attention Logits Post Processor Guided Decoding
Overlap Scheduler ---
CUDA Graph Yes ---
Attention Data Parallelism Yes Yes ---
Disaggregated Serving Yes Yes Yes ---
Chunked Prefill Yes Yes Yes Untested ---
MTP Yes Yes Yes Yes Untested ---
EAGLE-3(One Model Engine) Yes Yes Yes No Untested No ---
EAGLE-3(Two Model Engine) NO Yes Yes No Untested No No ---
Torch Sampler Yes Yes Yes Yes Yes Yes Yes Yes ---
TLLM C++ Sampler Yes Yes Yes Yes Yes No No No No ---
KV Cache Reuse Yes Yes Yes Untested Yes Untested Yes No Yes Yes ---
Slide Window Attention Yes Yes Yes Untested No Untested Untested Untested Yes Yes WIP ---
Logits Post Processor No Yes Yes No Untested No No No Yes Yes Yes Yes ---
Guided Decoding Yes Yes Yes No Yes No No No Yes Yes Yes Yes Yes ---