kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-02-21 18:25:20 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
bf4f7ad744
TensorRT-LLMs/tests/unittest/_torch/auto_deploy
History
Lucas Liebenwein be916b19e0
feat: [AutoDeploy] unfusing attention for native support (#3668) 
* [AutoDeploy] unfused streamlined attention + caching

Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>

* improved unit testing

Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>

* reviewer feedback

Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>

* some updates to attn_mask handling

Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>

* updated manual benchmarking and cudagraph capture

Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>

---------

Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>
2025-05-02 09:06:49 +08:00
..
_utils_test feat: [AutoDeploy] unfusing attention for native support (#3668) 2025-05-02 09:06:49 +08:00
integration chore: Rename nvsmall to nemotron nas (#3447) 2025-04-10 23:16:52 +08:00
unit feat: [AutoDeploy] unfusing attention for native support (#3668) 2025-05-02 09:06:49 +08:00