Bo Li 9ae705af1b perf: Add fused q_norm/k_norm/RoPE for Qwen3. (#4482) ... * Add Julien's origina kernel. Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com> * Get rid of UpdateKVCache functionality. Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com> * Add kernels. Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com> * Add torch OP. Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com> * Update cmake. Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com> * Torch OP must use double as argument dtype. Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com> * Add unittest. Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com> * Add unittest. Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com> * Fix misaligned access when head_dim=64. In this case, numElemsPerThread=2, numVecPerThread=0. But the store code incorrectly perform vectorized store, some threads (e.g., lane1) issue store to address that is not aligned to 64 bit. Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com> * Remove unroll (compiler can do that). Cleanup code. Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com> * Add switch for interleave. Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com> * Refactor vectorized load/store. Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com> * Implement is_neox. Result not correct yet. Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com> * Fix is_neox=True. Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com> * Add q_weight and k_weight. Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com> --------- Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>		2025-05-23 15:31:04 +08:00
..
batch_manager	[feat][TRTLLM-5018] Dis serving python runtime trt backend (#4243)	2025-05-22 22:01:06 -04:00
common	[feat][TRTLLM-5018] Dis serving python runtime trt backend (#4243)	2025-05-22 22:01:06 -04:00
cutlass_extensions/include/cutlass_extensions	chore: guardword clean for header file. (#4540)	2025-05-23 10:08:14 +08:00
executor	Agent interface impl for NIXL (#4125)	2025-05-22 09:09:41 +08:00
executor_worker	Update TensorRT-LLM (#2792)	2025-02-18 21:27:39 +08:00
kernels	perf: Add fused q_norm/k_norm/RoPE for Qwen3. (#4482)	2025-05-23 15:31:04 +08:00
layers	Feat: Variable-Beam-Width-Search (VBWS) part4 (#3979)	2025-05-12 22:32:29 +02:00
plugins	Fix bias shape in weightOnlyGroupwiseQuantMatmulPlugin for TRT workflow (#4348)	2025-05-16 10:02:30 +08:00
pybind	[feat][TRTLLM-5018] Dis serving python runtime trt backend (#4243)	2025-05-22 22:01:06 -04:00
runtime	fix[nvbug-5295425]: [TRTLLM-5385] fix race condition in MoeLoadBalancer (#4573)	2025-05-23 09:24:23 +08:00
testing	refactor: Move ModelSpec to core library (#3980)	2025-05-04 01:39:09 +08:00
thop	perf: Add fused q_norm/k_norm/RoPE for Qwen3. (#4482)	2025-05-23 15:31:04 +08:00
CMakeLists.txt	feat: NIXL interface integration (#3934)	2025-05-19 18:18:22 +08:00