kanshan/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-07-10 13:10:22 +00:00
Code Issues Packages Projects Releases Wiki Activity
9,951 Commits 613 Branches 6,783 Tags
082b326fc76f6e9bbb835b3920a3022bfdb6691c
Commit Graph

75 Commits

Author SHA1 Message Date
Martin Chang 082b326fc7 ggml-et: Initial ET backend (#24179) 
* ggml-et: Add performance logging

* ggml-et: Quants helpers

* ggml-et: Add MUL_MAT kernel

* ggml-et: Add ROPE kernel

* ggml-et: Add RMS_NORM kernel

* ggml-et: Add GLU kernel

* ggml-et: Add SOFT_MAX kernel

* ggml-et: Add GET_ROWS kernel

* ggml-et: Add CONT kernel

* ggml-et: Add SET_ROWS kernel

* ggml-et: Add MUL_MAT_ID kernel

* ggml-et: Build et kernels as part of ggml

* ggml-et: Embed kernels with fs fallback

* ggml-et: Build fixes

* ggml-et: Add MUL_MAT F32xF32 op

* ggml_et: Add MUL_MAT_ID op

* ggml-et: Disable offloading for debug

* ggml-et: Refactor out block ops

* ggml-et: ggml backend API changes

* ggml-et: Add RESHAPE/TRANSPOSE to supported

* ggml-et: Add CONT_F16

* ggml-et: Add supported ops doc

* gglm-et: Initial doc

* ggml-et: Remove  runtime import hacks

We can now import the runtime by a simple find_package(), so we
can cleanup the CMakeLists.txt.

* ggml-et: Fix GET_ROWS kernel

Fix lost batch dimension.

Also clean vibe-comments.

* ggml-et: Fix SET_ROWS kernel

Remove incorrect broadcasting guard.

* ggml-et: Use custom instruction for fp32->fp16

* ggml-et: Vectorize set_rows fp32->fp16

* ggml-et: Fix ROPE kernel (yarn)

ggml-et: fix et_logf

WIP: Fix ramp

WIP: fix ROPE!

* ggml-et: Better sinf

* ggml-et: Fix SOFT_MAX

Add `max_bias` and `sink` support.

* ggml-et: Fix CONT

Reorder from contiguous write to read with atomic stores.

* ggml-et: Fix elmap kernel

Remainder handlin

* ggml-et: Fix MUL_MAT MUL_MAT_ID remainders

* ggml-et: Fix ET-SOC reference

* ggml-et: Fix embed kernels scripts for old python

This allows GGML-ET to build on pre-3.8 python.

* Add sysemu support with compile time flag `-DGGML_ET_SYSEMU=ON` (#6)

* Example using ET-Soc-1 emulator configuration

Example usage:
```bash
cmake -B build -DGGML_CUDA=OFF -DGGML_ET=ON -DLLAMA_CURL=OFF -DGGML_CCACHE=ON
cmake --build build --config Release -j $(nproc)

time ./build/bin/test-backend-ops

./build/bin/llama-server \
    --model Qwen3-0.6B-Q8_0.gguf \
    --alias Qwen3-0.6B-Q8_0 \
    -fa 0 \
    --ctx-size 1024 \
    --no-warmup \
    --host 127.0.0.1 \
    --port 8080
```

* build: proper dep tracking for kernels

* support host using MOLD linker

* initial multi core GET_ROW F32 implementation

* vectorized q8 dequant

* wip: cland warning clenaups and initial logging refactor

* wip: message default message cleanup

* chore: message cleanups

* cmake cleanup

* migrate to use platform provided functions

* cmake back into subdir

* support et_print() in kernels

* fix: repair kernel building

* perf: operations run async by default

* debug: proper kernel dep tracking and error detection on kenrel launch

* fix: kernel binary dep tracking and fixing get_rows_f32 erroring

* perf: back to doing async kernel runs by default

* perf: vectorize and parallel device memset

* merge matmul work

* misc: align allocation and enable all offload

* misc: delete deadcode and respect memory limits

* fix: repair tensor debug print

* fix: loosen RMS_NORM op percision

* feat: Q4_0 GET_ROWS

* perf: FP32 MUL_MAT using TensorFMA

* update limitations

* perf: redue L1 load in compute_block_dot_product_q8_0

* feat: save kernel mapping (name to id) when profiling is enabled

* chore: memops cleanup

* perf: parallelize softmax by rows

* perf: vectorize 2nd phase of softmax

* perf: ban GET_ROWS from offloaded

* perf: vectorize and non-atomic for eltwise ops and sub support

* perf: vectorize normal rope

* perf: glu runs in parallel

* merge: manually merge saqib's work on kernel fixes

* perf: more vectorized RoPE

* perf: parallelize mul_mat_id

* perf: parallelize set_rows_f32

* perf: vectorize softmax

* feat: support kernel fusion and fuse RMS_NORM + MUL

* fix: mostly resolve test-backend-ops failure in SOFT_MAX and ROPE

* fix: bump max rope dims for gemma

* feat: GeGLU and SCALE support to fully offload Gemma

* perf: faster device memset

* feat: get_rows supporting Q4_K and avoid cont cache coherent issues

* better F32 MM

* feat: NORM for ET backend

* feat: SQR for ET backend

* feat: UNARY on ET

* feat: el_map support broadcasting for ET

* feat: SUM_ROWS in ET backend

* feat: more ops in ET backend

* feat: WKV* operators in ET backend

* perf: parallelize operators across cacheline instead of row

* perf: parallelize get_rows on cacheline

* wip: baseline FlashAttention for ET backend

* wip: enough FA and CPY f32->f16 to run llama 3.1 fully offloaded with FA on

* feat: f16 x f16 -> f32 MM using matrix engine

* wip: f16 FlashAttention using matrix engine

* wip: clean up

* feat: barriers

* perf: optimize FA_F16 in ET

* perf: vectorize pack_k_for_transpose16

* perf: prefetch next loop matrix tile

* perf: FlashAttention 2nd MM uses TensorFMA and optimizations

* cleanup: flashattention reorg

* perf: optimizations and fixes

* feat: L2SCP API and make FlashAttention support DV = 256 for gemma

* perf: parallelize norms beyond single row

* feat: GATED_DELTA_NET support and relaxed L2_NORM requirment

* feat: loosen RMS_NORM, NORM, ROPE contingous req too

* feat: repeat supports brocasting on dim 0 and loosen cont check

* feat: FILL and DIAG operator

* feat: loosen UNARY support chcek

* feat: TRI support

* feat: SOLVE_TRI support

* feat: basic SET support

* feat: loosen CONT req

* perf: fp16_to_fp32 use ASM

* feat: IMROPE support

* feat: PAD support

* feat: global barrier

* fix: view must live on the same backend as backing tensor

* feat: relax CONCAT in ET backend

* feat: dead simple CUMSUM implementation

* feat: basic SSM_CONV support

* feat: loosen CONCAT req

* feat: relax GATED_DELTA_NET and add SET support proper

* cleanup: cleanup LCM math

* feat: SWIGLU single input

* feat: SSM_SCAN support

* feat: el_map supports non aligned tensors in best effort

* feat: basic GROUP_NORM support

* feat: loosen MUL_MAT capablities slightly

* feat: loosen MUL_MAT and GET_ROWS and add IM2COL

* feat: special case for softmax 1x1x1x1

* feat: loosen SOFT_MAX req in ET backend

* fix: el_map unaligned acse fixes

* perf: optimize zero_acc_vec in flash_attn_ext_f16_me

* perf: use hart 1 for packing in MM and FA for FP16

* feat: kernel semaphore

* perf: better instruction sequence in FlashAttention

* fix: gated_delta_net with proper masking

* perf: better parallelization for GATED_DELTA_NET

* perf: parallelize SSM_CONV over nr

* perf: vectorize SSM_CONV

* perf: optimize MUL_MAT for q8

* feat: support Gemma 4

* fix: support multi-device

* feat: broader GLU support

* feat: unary ops supports view

* fix: repair fp16 MM using matrix engine

* perf: handle large N GEMV better

* perf: better q8_0 MM

* perf: better set_rows

* add back deleted files

* fix: repair after merge

* feat: POC version of uberkernel

* feat: RMS_NORM in uberkernel

* feat: add more kernels into usage

* chore: clean up uberkernel compilation

* perf: faster flash attention

* perf: opt flash attention for large seq length

* feat: loosen op bounds. clamp and mean support

* perf: vectorize ssm_scan

* perf: slightly faster FA

* perf: FlashAttention parallel MM and load

* perf: fuse Q8 MM and ADD

* feat: basic conv kernel for ET

* softMAx_test

* set_rows_f32

* get_rows and cont

* testing

* set_rows_exp

* Junk addition

* Narrowing the issue

* Update flash_attn_ext_f16_me.c

Focusing FA_ext_f16_me

* test

* Eviction updated

* Detailed cache eviction debug

* mulmat

* removeal of `BUILD_FOR_UBERKERNEL` flag

* cleaning...

* fix: balance FCC0 count

* feat: implement mul_mat and mul_mat_id for Q4_0 type

* optimize uberkernel plan upload

* add mul_mat q4 into uberkernel

* enable gating flush to just uberkernel

* update docs for ET

* update op support for ET

* et-backend: optimize Q4_0 and Q8_0 mul_mat_id row accumulations

* et-backend: specialize mul_mat_id kernels for Q4_0 and Q8_0

* et-backend: fix RoPE YaRN corr_dim formula and handle degenerate inputs

* test-backend-ops: add DeepSeek-V2-Lite RoPE test coverage

* et-backend: add Q4_0 mul_mat matrix-engine kernel using TensorFMA32

* et-backend: vectorize Q4_0 matrix-engine dequantization

* et-backend: support hybrid matrix/vector engine execution for Q4_0 mul_mat tail

* et-backend: run partial-N tiles on matrix engine for Q4_0 mul_mat

* et-backend: route Q4_0 mul_mat N < 53 to vecdot for better prefill latency

* Update uberkernel.c

* Update unary_f32.c

* gemma 4

* bisect gemma4: enable scale_f32 only

* bisect gemma4: +rms_norm_f32

* bisect gemma4: +rms_norm_mul_f32

* bisect gemma4: disable rms_norm_mul_f32 -- BREAKS OUTPUT

* bisect gemma4: +rope_f32 (skip rms_norm_mul)

* bisect gemma4: +el_map_f32

* bisect gemma4: +softmax_f32

* bisect gemma4: +get_rows_f32

* bisect gemma4: +glu_f32

* bisect gemma4: +mul_mat_f32 +mul_mat_f32_matrix_engine

* bisect gemma4: +mul_mat_f16 +mul_mat_f16_matrix_engine

* bisect gemma4: +mul_mat_Q8_0 +mul_mat_Q4_0

* bisect gemma4: +flash_attn_ext_f32 +flash_attn_ext_f16_me

* bisect gemma4: +mul_mat_id_f32

* bisect gemma4: +sum_rows_f32

* bisect gemma4: +cont_f16

* bisect gemma4: +fill_f32

* bisect gemma4: +unary_f32 (all ops re-enabled except rms_norm_mul)

* Update rms_norm_mul_f32.c

* bisect2 gemma4 n64: +scale_f32 only

* bisect2 gemma4 n64: +rms_norm_f32 +rope_f32

* bisect2 gemma4 n64: +rms_norm_mul_f32 (with ET_UBERKERNEL eviction fix)

* bisect2 gemma4 n64: +el_map +get_rows +glu +softmax (skip rms_norm_mul)

* bisect2 gemma4 n64: all ops enabled except rms_norm_mul

* bisect2 n64: test unary+cont+fill+sum_rows (no mul_mat/flash_attn)

* bisect2 n64: +mul_mat_f32 +mul_mat_f32_matrix_engine

* bisect2 n64: +mul_mat_f16 +mul_mat_f16_matrix_engine

* bisect2 n64: +mul_mat_Q8_0 +mul_mat_Q4_0

* bisect2 n64: +mul_mat_Q8_0 only (disable Q4_0)

* bisect2 n64: +mul_mat_Q4_0 only (Q8_0 breaks)

* bisect2 n64: +mul_mat_id +flash_attn_ext (skip Q8_0)

* run-3: matmul + rms_norm_mul

* run-4

* Revert "run-4"

* run5

* changes after cleanup

* cleanup before upstream

* restrict changes into ET backend

* move kernel embedding from Python to CMake

* move uberkernel gen into CMake

* apply clang format

* update CMake style

* update to match C and C++ style

* use source ggml and quant headers instead of ET's

* MROPE support

* absorb view ops into same branch as none

* fix bad rebase

* add marty1885 to codeowners

* oops

* remove redundant newline

* fix CI editor warnings

---------

Co-authored-by: Vidas <vidas@nuolat.lt>
Co-authored-by: Gianluca Guida <glguida@tlbflush.org>
Co-authored-by: Gianluca Guida <gianluca@nekko.ai>
Co-authored-by: ubergarm <leimgrub@gmail.com>
Co-authored-by: SaqibAkram-10xE <saqib.akram@10xengineers.ai>
Co-authored-by: Rehan Qasim <rehan.qasim@10xengineers.ai>
 2026-07-10 12:38:34 +08:00
Neo Zhang 108f186d17 [SYCL] fix unsupported UT cases of CONT & CPY (#25231) 
* fix unsupported UT cases of CONT & CPY

* update ops.md

* rm unused head file
 2026-07-07 12:20:52 +03:00
Neo Zhang 47e1de77aa [SYCL] support op col2im_1d (#25264) 
* support op col2im_1d

* update ops.md

* rm unused words

* update for bf16

* optimize 1%-11% as the review comments

* fix the format issue

* update as the review comments
 2026-07-07 11:07:46 +03:00
Neo Zhang 55edb2de44 [SYCL] support OP cross_entropy_loss, cross_entropy_loss_back (#25236) 
* support OP cross_entropy_loss, cross_entropy_loss_back

* correct format issue
 2026-07-07 10:48:50 +03:00
Neo Zhang 9e5ef0dbb1 sycl : enhance argsort to support all UT cases (#25125) 2026-07-07 10:39:29 +03:00
Neo Zhang 6f1034b32a [SYCL] support OPs: conv_2d, conv_2d_dw, conv2d_transpose (#24600) 
* fix conflict

* fix format issue, rename

* rm debug code

* correct the file name
 2026-06-18 09:40:03 +03:00
Neo Zhang d1759e4156 [SYCL] Add conv_3d (#24691) 
* add conv_3d

* optimize

* update ops.md

* restore test script

* rm unused code

* rm copyright notes
 2026-06-17 17:20:01 +03:00
Neo Zhang 58728bdbf0 sycl : Enable to support fp16 by OPs: SQR, SQRT, LOG, SIN, COS, CLAMP (#24692) 2026-06-17 08:58:03 +03:00
Neo Zhang fdd109883d [SYCL] Support OP EXPM1, support all UT cases of FLOOR, TRUNC, ROUND (#24363) 
* support OP EXPM1, support all UT cases of FLOOR, TRUNC, ROUND

* fix conflict

* rebase, support new UT case of repeat, concat
 2026-06-16 08:34:29 +03:00
Neo Zhang 8872ab5467 sycl : enhance set_rows to support q1_0, mxfp4, nvfp4 (#24564) 2026-06-15 10:01:40 +03:00
Neo Zhang 987fbd821d [SYCL] add to support pool_1d, move pool_1d/2d code to pool.cpp/hpp (#24584) 
* add to support pool_1d, move pool_1d/2d code to pool.cpp/hpp

* update ops.md
 2026-06-15 10:01:07 +03:00
Jeff Bolz 1a7718b4c5 vulkan: support non-contig unary/glu ops (#24215) 
* vulkan: support non-contig unary/glu ops

Change unary/glu ops to pass in all strides and use fastdiv for the index
calculation. Put all unary ops in one file, similar to glu, to share the
code. codex went ahead and added expm1 without me asking, but I had to
make it do a real precision analysis rather than just making stuff up.

unary.comp initially couldn't use generic_unary_head because there wasn't
space for xielu's additional constants. Fixing this required packing the
fastdiv 'L' values.

* attempt to workaround compiler bug

* resolve conflict from #23991

* use expm1
 2026-06-13 08:44:15 -05:00
Neo Zhang a51142497a [SYCL] Support Q4_1, Q5_0, Q5_1 in Flash-attention (#23812) 
* support Q4_1, Q5_0, Q5_1

* update ut case
 2026-06-01 09:53:53 +03:00
Neo Zhang 4162522688 [SYCL] Add more types in GET_ROWS OP (#23710) 
* add to support Q1_0, NVFP4, IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS, IQ1_S, IQ1_M, IQ3_S, IQ4_NL, IQ4_XS, I32, MXFP4, Q2_K, Q3_K, Q5_K, and Q6_K in GET_ROWS OP

* correct the link
 2026-06-01 09:53:04 +03:00
Masashi Yoshimura 927dada6c9 ggml-webgpu: Enables running gpt-oss-20b (#22906) 
* Enable to run gpt-oss-20b and refactor mulmat-q

* disable test-backend-ops in ubuntu-24-webgpu
 2026-05-12 07:27:40 -07:00
Neo Zhang 7d442abf5c [SYCL] Add OP im2col_3d (#22903) 
* add im2col_3d

* format code

* update the ops.md
 2026-05-11 08:01:47 +03:00
Intel AI Get-to Market Customer Success and Solutions ad09224658 sycl: add FILL, CUMSUM, DIAG, SOLVE_TRI, SSM_SCAN, GATED_DELTA_NET (#22149) 
* sycl: add FILL, CUMSUM, DIAG, SOLVE_TRI, SSM_SCAN, GATED_DELTA_NET

Signed-off-by: Chun Tao <chun.tao@intel.com>

* Fix abort during test-backend-ops

Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>

* Regenerate ops.md

Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>

* Add scope_dbg_print to newly added SYCL ops.

Also add scope_dbg_print to existing ssm_conv op.

Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>

---------

Signed-off-by: Chun Tao <chun.tao@intel.com>
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
Co-authored-by: Chun Tao <chun.tao@intel.com>
Co-authored-by: Todd Malsbary <todd.malsbary@intel.com>
 2026-05-07 18:51:33 +03:00
Reese Levine dd2914dc81 ggml-webgpu: support for SSM_SCAN and disable set_rows error checking (#22327) 
* Implement ssm_scan

* Remove blocking in graph_compute and check for set rows

* Fix bindings

* Update op support
 2026-04-25 09:18:15 +03:00
Kusha Gharahi ae2d34899e metal: Implement ROLL op (#21946) 
* nix: support unified apple-sdk

* Impl roll op for Metal

* Revert "nix: support unified apple-sdk"

This reverts commit abfa473360.

* update ops.md

* update op docs
 2026-04-16 11:54:37 +03:00
Masashi Yoshimura d0a6dfeb28 ggml-webgpu: Add the support of MUL_MAT_ID (#21147) 
* Add mul_mat_id support to WebGPU

* Apply suggestion from @reeselevine

---------

Co-authored-by: Reese Levine <reeselevine1@gmail.com>
 2026-04-06 13:08:46 -07:00
Vishal Singh f1ac84119c ggml-zendnn : add MUL_MAT_ID op support for MoE models (#21315) 
* ggml-zendnn : add MUL_MAT_ID op support for MoE models
- Add MUL_MAT_ID op acceleration for Mixture-of-Experts models
- MUL_MAT_ID op fallback to CPU backend if total experts > 32
- Point ZenDNN lib to latest bits ZenDNN-2026-WW13

* ggml-zendnn : add braces to sgemm failure condition for consistency

Co-authored-by: Aaron Teo <taronaeo@gmail.com>

---------

Co-authored-by: Aaron Teo <taronaeo@gmail.com>
 2026-04-03 12:19:08 +03:00
Seyoung Jeong 6d99b44c7e docs : fix Metal backend op support status in ops.md (#20779) 
Regenerate docs/ops/Metal.csv using test-backend-ops on Apple M5
and rebuild docs/ops.md via scripts/create_ops_docs.py.

Five ops were incorrectly marked as not supported (❌) for Metal:
- DIAG:           ❌ → ✅
- POOL_1D:        ❌ → ✅
- SET:            ❌ → ✅
- SOLVE_TRI:      ❌ → ✅
- GATED_DELTA_NET:❌ → 🟡 (partial, depends on head_size % 32)
 2026-03-20 11:06:38 +02:00
Reese Levine c1258830b2 ggml webgpu: ops support for qwen3.5 (SET, TRI_SOLVE, SSM_CONV, GATED_DELTA_NET) + GET_ROWS optimization (#20687) 
* Implement l2_norm, set, tri

* Add DIAG/SOLVE_TRI

* Add SSM_CONV

* Better get_rows and gated_delta_net to support qwen3.5

* Clean up, update ops.md

* Fix binding_index type for wasm

* Fix read write annotations

* cleanups
 2026-03-19 08:45:28 -07:00
Masashi Yoshimura 509a31d00f ggml-webgpu: Update the RMS_NORM preprocessor and add L2_NORM (#20665) 
* Update the preprocessor of RMS_NORM and add L2_NORM.

* Fix the name of rms_norm to row_norm.
 2026-03-18 21:08:59 -07:00
Masashi Yoshimura ea01d196d7 ggml-webgpu: Add supports for DIAG and TRI (#20664) 
* Add supports for DIAG and TRI.

* Remove extra ttype and add a comment for TRI op.
 2026-03-18 21:08:35 -07:00
Neo Zhang b6c83aad55 [SYCL] ehance UPSCALE to support all UT cases (#20637) 
* [SYCL] ehance UPSCALE to support more cases

* rm test case result of SYCL1
 2026-03-17 10:01:52 +08:00
Neo Zhang a93c0ef0fa add op gated_delta_net (#20455) 2026-03-14 22:01:57 +08:00
Masashi Yoshimura f2ab047f27 ggml-webgpu: Add supports for GGML_OP_REPEAT (#20230) 
* Add GGML_OP_REPEAT to webgpu backend.

* Add i16 support for GGML_OP_REPEAT.
 2026-03-11 14:40:36 -07:00
Neo Zhang 0cec84f999 fix op rope, add rope_back (#20293) 2026-03-11 09:53:34 +08:00
a3894281 0f1e9d14cc docs: update CPU backend ops to mark POOL_1D as supported (#20304) 2026-03-10 21:31:24 +08:00
Bertay Eren 0beb8db3a0 ggml-vulkan: add SGN operator, auto-generate Vulkan.csv and ops.md (#20219) 2026-03-09 07:24:16 +01:00
GiantPrince d088d5b74f ggml-vulkan: Add ELU op support (#20183) 
* ggml-Vulkan: add ELU support

* ggml-Vulkan: remove extra spaces and variables

* ggml-Vulkan: fix format issue

* ggml-Vulkan: fix format issue

* fix whitespace issue

* Update Vulkan.csv and ops.md
 2026-03-08 12:38:17 +01:00
Neo Zhang 213c4a0b81 [SYCL] supprt Flash Attention for fp32/fp16/Q4/Q5/Q8 (#20190) 
* support flash-attention for fp32/fp16/Q4/Q5/Q8

* rm warining

* update for JIT
 2026-03-08 12:00:07 +08:00
Masashi Yoshimura 541bf37622 Add concat op to webgpu. (#20068) 2026-03-04 11:19:00 -08:00
Masashi Yoshimura 11c325c6e0 ggml-webgpu: Add unary op (SQR, SQRT, SIN, COS) support. (#19700) 
* ggml-webgpu: Add unary op (SQR, SQRT, SIN, COS) support.

* Fix to cast the src value to f32 before sin/cos computing.
 2026-02-19 09:18:30 -07:00
Nechama Krashinski 537eadb1b9 sycl: add F16 support for GGML_OP_CEIL (#19306) 
* Fix SYCL CEIL operator

* sycl: implement GGML_OP_CEIL
 2026-02-06 23:13:44 +08:00
Tamar 4d5e972673 sycl: implement GGML_OP_TOP_K (#19242) 2026-02-02 21:05:51 +08:00
s8322 1025fd2c09 sycl: implement GGML_UNARY_OP_SOFTPLUS (#19114) 
* sycl: add softplus unary op implementation

* sycl: add softplus unary op implementation

* docs(ops): mark SYCL SOFTPLUS as supported

* docs: update SYCL status for SOFTPLUS
 2026-01-30 12:01:38 +08:00
RachelMantel c7358ddf64 sycl: implement GGML_OP_TRI (#19089) 
* sycl: implement GGML_OP_TRI

* docs: update ops.md for SYCL TRI

* docs: regenerate ops.md

* docs: update SYCL support for GGML_OP_TRI
 2026-01-30 12:00:49 +08:00
Neo Zhang d4964a7c66 sycl: fix norm kernels: l2_norm, group_norm, rms_norm by remove assert to support more cases (#19154) 
Co-authored-by: Neo Zhang Jianyu <jianyu.zhang@intel.com>
 2026-01-29 09:20:22 +08:00
Reese Levine a89002f07b ggml webgpu: support for backend sampling (#18880) 
* ggml webgpu: add SOFTPLUS unary operator

Implements SOFTPLUS (log(1 + exp(x))) with f16/f32 support. Uses f32
precision for intermediate calculations to prevent f16 overflow.

* Add shader implementation and 4 variants (f32/f16, inplace/non-inplace)
* Register pipelines and device support
* Follow Vulkan backend numerical stability pattern

* ggml webgpu: add EXPM1 unary operator

Implements EXPM1 (exp(x) - 1) with f16/f32 support.

* Add shader implementation and 4 variants (f32/f16, inplace/non-inplace)
* Register pipelines and device support

* ggml webgpu: add FLOOR unary operator

Implements FLOOR (rounds down to nearest integer) with f16/f32 support.

* Add shader implementation and 4 variants (f32/f16, inplace/non-inplace)
* Register pipelines and device support

* ggml webgpu: add CEIL unary operator

Implements CEIL (rounds up to nearest integer) with f16/f32 support.

* Add shader implementation and 4 variants (f32/f16, inplace/non-inplace)
* Register pipelines and device support

* ggml webgpu: add ROUND unary operator

Implements ROUND (rounds to nearest integer) with f16/f32 support.

* Add shader implementation and 4 variants (f32/f16, inplace/non-inplace)
* Register pipelines and device support

* ggml webgpu: add TRUNC unary operator

Implements TRUNC (truncates towards zero) with f16/f32 support.

* Add shader implementation and 4 variants (f32/f16, inplace/non-inplace)
* Register pipelines and device support

* docs : update WebGPU support for unary operators (FLOOR, CEIL, ROUND, TRUNC, EXPM1, SOFTPLUS)

* Updates to webgpu get_memory

* Add argmax

* Add argmax,cumsum,sum,sum_rows

* Add necessary CPY/GET_ROWS operators

* Support for argsort using multi-pass strategy

* Update set_rows for i32 indices, move to pre-wgsl

* Port unary operators to pre-wgsl and support FILL

* Implement PAD

* Add support for top-k

* clean up, scope pipeline init mutex

* fix newline

* Add support for log

* Update LOG for better precision, and ops doc

---------

Co-authored-by: Abhijit Ramesh <abhijitramesh2k@gmail.com>
 2026-01-16 16:12:43 -08:00
hipudding 6ba6a3c76f docs : update ops.md for CANN backend (#18654) 2026-01-16 13:32:17 +01:00
Aaron Teo 2656c0d265 docs(ggml): update backend ops (#18734) 
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
 2026-01-10 18:48:17 +08:00
nwyin e443fbcfa5 ggml webgpu: add CEIL operation support (#18605) 
* ggml-webgpu: add CEIL operation support

      Add support for the CEIL unary operation in the WebGPU backend:
      - Add CEIL_FUNC shader template in unary_op.wgsl
      - Add 4 shader variants (f32, f16, inplace versions)
      - Initialize CEIL pipelines in ggml-webgpu.cpp
      - Register CEIL in supports_op function

* docs: update WebGPU ops support for CEIL
 2026-01-05 11:38:57 -08:00
gatbontonpc 9a6369bb60 metal : add count_equal op (#18314) 
* add count equal for metal

* remove trailing whitespace

* updated doc ops table

* changed shmem to i32

* added multi tg and templating

* removed BLAS support from Metal docs

* Apply suggestions from code review

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* add memset to set dst to 0

* metal : cleanup

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-12-31 10:39:48 +02:00
Neo Zhang Jianyu 4aced7a631 [SYCL] Support gpt-oss by OPs add-id, mul_mat for mxfp4, swiglu_oai (#17826) 
* support gpt-oss GPU by OP add-id, mul_mat for mxfp4, swiglu_oai, fix warning

* fix fault ut case, update ops.md

* rebase, fix format issue
 2025-12-15 10:35:15 +08:00
lhez 2d2e1030e3 docs : update opencl ops (#17904) 2025-12-10 15:20:00 +01:00
Sigbjørn Skjæret 63391852b0 docs : update cpu and cuda ops (#17890) 
* update cuda ops

* update CPU as well
 2025-12-09 23:31:29 +01:00
Vishal Singh 017761daf5 ggml-zendnn : add ZenDNN backend for AMD CPUs (#17690) 
* ggml-zennn: add ZenDNN backend support

* ggml-zendnn : address ZenDNN backend review fixes and suggestions

* docs : apply blockquote syntax to ZenDNN docs

---------

Co-authored-by: Manoj Kumar <mkumar@zettabolt.com>
 2025-12-07 00:13:33 +08:00
Reese Levine fd57b24c0f ggml webgpu: unary op suppport, code refactoring, ops support (#17764) 
* Squashed commit of the following:

commit b3c6bf4b0450d8d452b934df27a0fb7cb53cd755
Author: Abhijit Ramesh <abhijitramesh2k@gmail.com>
Date:   Mon Dec 1 18:29:00 2025 -0800

    ggml webgpu: fix xielu parameter passing (#11)

    The XIELU operation was incorrectly using static_cast to convert
    float parameters to uint32_t, which converted numeric values instead
    of preserving IEEE 754 bit patterns. This caused incorrect values
    to be interpreted by the GPU shader.

    * Use reinterpret_cast to preserve float bit patterns when passing
      through uint32_t params buffer
    * Update WGSL shader parameter types from u32 to f32
    * Re-enable XIELU support (was disabled due to numerical issues)

    Fixes NMSE test failures for XIELU operation on WebGPU backend.

commit 5ca9b5e49e
Author: neha-ha <137219201+neha-ha@users.noreply.github.com>
Date:   Tue Nov 18 12:17:00 2025 -0800

    Refactored pipelines and workgroup calculations (#10)

    * refactored pipelines

    * refactored workgroup calculation

    * removed commented out block of prior maps

    * Clean up ceiling division pattern

    ---------

    Co-authored-by: Neha Abbas <nehaabbas@eduroam-169-233-141-223.ucsc.edu>
    Co-authored-by: Reese Levine <reeselevine1@gmail.com>

Author: James Contini <jamescontini@gmail.com>
Date:   Wed Oct 29 23:13:06 2025 -0700

    formatted embed wgsl and ggml-webgpu.cpp

commit e1f6baea31
Author: James Contini <jamescontini@gmail.com>
Date:   Wed Oct 29 23:08:37 2025 -0700

    implemented REPL_Template support and removed bug in unary operators kernel

commit 8c70b8fece
Author: James Contini <jamescontini@gmail.com>
Date:   Wed Oct 15 16:14:20 2025 -0700

    responded and dealt with PR comments

commit f9282c660c
Author: James Contini <jamescontini@gmail.com>
Date:   Sun Oct 12 13:41:41 2025 -0700

    removed unnecesarry checking if node->src[1] exists for unary operators

commit 4cf28d7dec
Author: James Contini <jamescontini@gmail.com>
Date:   Sun Oct 12 13:32:45 2025 -0700

    All operators (inlcluding xielu) working

commit 74c6add176
Author: James Contini <jamescontini@gmail.com>
Date:   Fri Oct 10 13:16:48 2025 -0700

    fixed autoconfig

commit 362749910b
Author: James Contini <jamescontini@gmail.com>
Date:   Fri Oct 10 13:10:46 2025 -0700

    removed vestigial files

commit cb08583337
Author: James Contini <jamescontini@gmail.com>
Date:   Fri Oct 10 12:59:32 2025 -0700

    abides by editor-config

commit 5360e2852a
Author: James Contini <jamescontini@gmail.com>
Date:   Fri Oct 10 12:45:57 2025 -0700

    rms_norm double declaration bug atoned

commit 7b09baa4aa
Merge: 8a6ec843 74b8fc17
Author: James Contini <jamescontini@gmail.com>
Date:   Fri Oct 10 11:50:03 2025 -0700

    resolving merge conflicts

commit 8a6ec843a5
Author: James Contini <jamescontini@gmail.com>
Date:   Wed Oct 8 18:06:47 2025 -0700

    unary operators pass ggml tests

commit c3ae38278a
Author: James Contini <jamescontini@gmail.com>
Date:   Wed Oct 1 16:22:40 2025 -0700

    neg passes backend test

commit aa1c9b2f88
Author: James Contini <jamescontini@gmail.com>
Date:   Tue Sep 30 23:55:27 2025 -0700

    neg f16xf32xip builds and runs, havent actually ran a model that uses neg kernel yet though

Co-authored-by: James Contini <jamescontini@gmail.com>
Co-authored-by: Neha Abbas <neabbas@ucsc.edu>
Co-authored-by: Abhijit Ramesh <abhijitramesh2k@gmail.com>

* Remove extra code and format

* Add ops documentation (finally)

* Update ggml/src/ggml-webgpu/wgsl-shaders/embed_wgsl.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

---------

Co-authored-by: James Contini <jamescontini@gmail.com>
Co-authored-by: Neha Abbas <neabbas@ucsc.edu>
Co-authored-by: Abhijit Ramesh <abhijitramesh2k@gmail.com>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2025-12-05 12:25:51 -08:00