kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
b8818b45be
TensorRT-LLMs/tensorrt_llm
History
Chang Liu b8818b45be
fix: llama4: address couple of issues in llama4 attention module (#3491) 
* fix attn module for llama4

* Address comments

* Rebase to accommodate latest attn refactor and refactor l4attn

* Remove aux_stream from classic attn

* Use RMSNorm for L2Norm

* Update tensorrt_llm/_torch/models/modeling_llama.py

Co-authored-by: hlu1 <14827759+hlu1@users.noreply.github.com>
Signed-off-by: Chang Liu <lc9114@gmail.com>

* Add typing informations for _attn_qkv

* Remove redundant comment

* Simplify llama4 DecoderLayer logic

---------

Signed-off-by: Chang Liu <lc9114@gmail.com>
Co-authored-by: hlu1 <14827759+hlu1@users.noreply.github.com>
2025-04-18 01:54:59 +00:00
..
_torch fix: llama4: address couple of issues in llama4 attention module (#3491) 2025-04-18 01:54:59 +00:00
auto_parallel chore: remove usernames from comments (#3291) 2025-04-05 13:44:28 +08:00
bench feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
commands Use llm.tokenizer in OpenAIServer (#3199) 2025-04-08 14:55:02 +08:00
evaluate feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
executor fix: Use hmac authentication for pickle encryption (#3384) 2025-04-17 00:40:13 +08:00
inputs feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
layers feat: Add FP8 support for SM 120 (#3248) 2025-04-14 16:05:41 -07:00
llmapi fix: LLM API _hf_model_dir for non-cached case (#3562) 2025-04-16 10:39:34 +08:00
models feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
plugin Update TensorRT-LLM (#2936) 2025-03-18 21:25:19 +08:00
quantization feat: adding multimodal (only image for now) support in trtllm-bench (#3490) 2025-04-18 07:06:16 +08:00
runtime test: Fix breaking Phi3 multimodal tests (#3544) 2025-04-15 08:02:34 +08:00
scaffolding feat: Make scaffolding Controller more generic #3408 (#3416) 2025-04-12 21:35:38 +08:00
serve chore: Unify Python NVTX call (#3450) 2025-04-15 23:25:36 +08:00
tools test: Fix breaking Phi3 multimodal tests (#3544) 2025-04-15 08:02:34 +08:00
__init__.py Update (#2978) 2025-03-23 16:39:35 +08:00
_common.py Update (#2978) 2025-03-23 16:39:35 +08:00
_ipc_utils.py fix: Proper error bubbling for PyExecutor (#3321) 2025-04-15 14:49:46 +08:00
_utils.py chore: Unify Python NVTX call (#3450) 2025-04-15 23:25:36 +08:00
builder.py chore: remove usernames from comments (#3291) 2025-04-05 13:44:28 +08:00
disaggregated_params.py Update TensorRT-LLM (#2936) 2025-03-18 21:25:19 +08:00
functional.py Cache sin cos in model instead of global LRU cache. (#3378) 2025-04-14 11:19:09 +08:00
graph_rewriting.py Update TensorRT-LLM (#2755) 2025-02-11 03:01:00 +00:00
logger.py Update TensorRT-LLM (#2873) 2025-03-11 21:13:42 +08:00
lora_manager.py Feat/ Integrate peftCacheManager in PyExecutor creation (#3372) 2025-04-15 15:14:43 +08:00
mapping.py chore: unify pp_layers helpers (#3429) 2025-04-15 04:49:17 +08:00
module.py Update (#2978) 2025-03-23 16:39:35 +08:00
network.py chore: remove usernames from comments (#3291) 2025-04-05 13:44:28 +08:00
parameter.py Update TensorRT-LLM (#2873) 2025-03-11 21:13:42 +08:00
profiler.py Update TensorRT-LLM (#2936) 2025-03-18 21:25:19 +08:00
prompt_adapter_manager.py Update TensorRT-LLM (#2333) 2024-10-15 15:28:40 +08:00
python_plugin.py Update TensorRT-LLM (#2755) 2025-02-11 03:01:00 +00:00
sampling_params.py v1.2 (#3082) 2025-03-26 23:31:29 +08:00
top_model_mixin.py Update TensorRT-LLM (#2053) 2024-07-30 21:25:01 +08:00
version.py chore: bump version to 0.20.0rc0 (#3561) 2025-04-16 11:41:21 +08:00