kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-22 03:35:00 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
b168adba70
TensorRT-LLMs/tensorrt_llm/_torch/modules
History
Zhihan Jiang 8300218d21
feat: support llama4 nope layers; support FP8 checkpoint loading; (#3382) 
* Enable NOPE, Fix a rotary embedding bug for gptj_stype_rope, Address PR comment, Properly skip the rotary_embdding for Llama4 ROPE layers

* Add support for FP8 checkpoint, Fix ckpt weighting loading for FP8

* Temporarily disable min_latency_mode for llama4

---------

Co-authored-by: Yilin Fan <yilinf@nvidia.com>
Co-authored-by: Sharan Chetlur <116769508+schetlur-nv@users.noreply.github.com>
2025-04-10 10:16:42 -07:00
..
__init__.py Update TensorRT-LLM (#2755) 2025-02-11 03:01:00 +00:00
attention.py [MLA] Deallocate tensors after use (#3286) 2025-04-09 21:36:07 -07:00
decoder_layer.py Update TensorRT-LLM (#2755) 2025-02-11 03:01:00 +00:00
embedding.py Update (#2978) 2025-03-23 16:39:35 +08:00
fused_moe.py feat: support llama4 nope layers; support FP8 checkpoint loading; (#3382) 2025-04-10 10:16:42 -07:00
gated_mlp.py lora_tests (#3201) 2025-04-09 18:06:52 +03:00
linear.py lora_tests (#3201) 2025-04-09 18:06:52 +03:00
logits_procesor.py Update (#2978) 2025-03-23 16:39:35 +08:00
mamba.py Update TensorRT-LLM (#2820) 2025-02-25 21:21:49 +08:00
mlp.py lora_tests (#3201) 2025-04-09 18:06:52 +03:00
rms_norm.py Add Llama 4 (#3302) 2025-04-09 03:35:21 +08:00
rotary_embedding.py feat: support llama4 nope layers; support FP8 checkpoint loading; (#3382) 2025-04-10 10:16:42 -07:00