mirror of
https://github.com/NVIDIA/TensorRT-LLM.git
synced 2026-02-14 15:03:48 +08:00
ec723fa993
* add test to map flashinfer rope op with triton custom rope ops and pytorch rope in fused_mha Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com> * add rope matcher and unit tests Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com> * capture cos and sin from graph Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com> * revert fuse_mha op change Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com> * minor update to address comment and remove redundant unit test Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com> * move view and transpose into graph nodes and update unit test to test custom op directly Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com> * move view into custom op, update bfs with bound, update custom op return type to be half precision Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com> * custom op update to support 3D input Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com> * handle bnsd and bsnd format, update tests, handle 3D cos/sin input to the custom op Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com> * add llama4 rope test, update custom op with is_neox flag Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com> * add llama4 style rope to matcher and update unit test Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com> * separate into two transformations Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com> * fix when num_head != num_kv_head; add support for cached position_ids and cos_sin_cache in graph; update unit tests Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com> * minor update, cache locally and propagate meta info of qk nodes Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com> * minor: fix cos_sin_cache not float Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com> * minor: move cache into matcher Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com> --------- Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| auto_deploy | ||
| compilation | ||
| modeling | ||
| modules/tests_lora_modules | ||
| multi_gpu | ||
| multi_gpu_modeling | ||
| speculative | ||
| thop | ||
| deep_gemm_tests.py | ||
| helpers.py | ||
| pattern_watcher.py | ||
| test_attention_no_cache.py | ||
| test_attention.py | ||
| test_autotuner.py | ||
| test_flashinfer_attention.py | ||
| test_flashinfer_star_attn.py | ||
| test_fp4_bmm_quantize.py | ||
| test_fp4_gemm_quantize.py | ||
| test_fp4_linear.py | ||
| test_fp8_block_scale_gemm.py | ||
| test_fp8_linear.py | ||
| test_fp8_quantize.py | ||
| test_fused_moe.py | ||
| test_moe_routing.py | ||
| test_moe.py | ||
| test_overlap_scheduler_input.json | ||
| test_overlap_scheduler.py | ||
| test_pytorch_model_engine.py | ||
| test_resource_manager.py | ||
| test_vanilla_attention.py | ||