mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-06-28 15:20:20 +00:00

Files

T

Jeff Bolz 1a7718b4c5 vulkan: support non-contig unary/glu ops (#24215 )

* vulkan: support non-contig unary/glu ops

Change unary/glu ops to pass in all strides and use fastdiv for the index
calculation. Put all unary ops in one file, similar to glu, to share the
code. codex went ahead and added expm1 without me asking, but I had to
make it do a real precision analysis rather than just making stuff up.

unary.comp initially couldn't use generic_unary_head because there wasn't
space for xielu's additional constants. Fixing this required packing the
fastdiv 'L' values.

* attempt to workaround compiler bug

* resolve conflict from #23991

* use expm1

2026-06-13 08:44:15 -05:00

11 KiB

Raw Permalink Blame History

GGML Operations

List of GGML operations and backend support status.

How to add a backend to this table:

Run test-backend-ops support --output csv with your backend name and redirect output to a csv file in docs/ops/ (e.g., docs/ops/CUDA.csv)
Regenerate /docs/ops.md via ./scripts/create_ops_docs.py

Legend:

✅ Fully supported by this backend
🟡 Partially supported by this backend
❌ Not supported by this backend

Operation	BLAS	CANN	CPU	CUDA	MTL	OpenCL	SYCL	Vulkan	WebGPU	ZenDNN	zDNN
ABS	❌	✅	✅	🟡	✅	❌	✅	✅	✅	❌	❌
ACC	❌	✅	✅	✅	✅	❌	🟡	✅	❌	❌	❌
ADD	❌	✅	✅	✅	🟡	✅	✅	✅	✅	❌	❌
ADD1	❌	✅	✅	✅	❌	❌	❌	❌	❌	❌	❌
ADD_ID	❌	❌	✅	✅	✅	✅	✅	✅	✅	❌	❌
ARANGE	❌	✅	✅	✅	✅	❌	✅	✅	❌	❌	❌
ARGMAX	❌	✅	✅	✅	✅	❌	✅	✅	✅	❌	❌
ARGSORT	❌	✅	✅	✅	✅	🟡	🟡	✅	✅	❌	❌
CEIL	❌	❌	✅	🟡	✅	❌	✅	✅	✅	❌	❌
CLAMP	❌	✅	✅	✅	✅	🟡	🟡	🟡	✅	❌	❌
CONCAT	❌	✅	✅	🟡	✅	🟡	✅	✅	✅	❌	❌
CONT	❌	🟡	✅	✅	✅	🟡	🟡	✅	🟡	❌	❌
CONV_2D	❌	❌	✅	✅	✅	✅	❌	✅	✅	❌	❌
CONV_2D_DW	❌	❌	✅	✅	❌	❌	❌	✅	❌	❌	❌
CONV_3D	❌	❌	✅	❌	✅	❌	❌	❌	❌	❌	❌
CONV_TRANSPOSE_1D	❌	✅	✅	✅	✅	❌	✅	✅	❌	❌	❌
CONV_TRANSPOSE_2D	❌	❌	✅	✅	✅	❌	❌	✅	❌	❌	❌
COS	❌	✅	✅	✅	✅	❌	🟡	🟡	✅	❌	❌
COUNT_EQUAL	❌	✅	✅	✅	✅	❌	✅	✅	❌	❌	❌
CPY	❌	🟡	🟡	🟡	🟡	🟡	🟡	🟡	🟡	❌	❌
CROSS_ENTROPY_LOSS	❌	✅	✅	✅	❌	❌	❌	❌	❌	❌	❌
CROSS_ENTROPY_LOSS_BACK	❌	❌	✅	✅	❌	❌	❌	❌	❌	❌	❌
CUMSUM	❌	❌	✅	✅	✅	❌	✅	✅	✅	❌	❌
DIAG	❌	❌	✅	✅	✅	❌	✅	✅	✅	❌	❌
DIAG_MASK_INF	❌	✅	✅	✅	❌	🟡	✅	✅	❌	❌	❌
DIV	❌	✅	✅	✅	🟡	✅	✅	✅	✅	❌	❌
DUP	❌	✅	✅	🟡	🟡	🟡	✅	✅	❌	❌	❌
ELU	❌	✅	✅	🟡	✅	❌	✅	✅	✅	❌	❌
EXP	❌	✅	✅	🟡	✅	❌	✅	✅	✅	❌	❌
EXPM1	❌	❌	✅	🟡	✅	❌	❌	✅	✅	❌	❌
FILL	❌	❌	✅	✅	✅	❌	✅	✅	✅	❌	❌
FLASH_ATTN_EXT	❌	🟡	✅	🟡	🟡	🟡	🟡	🟡	🟡	❌	❌
FLOOR	❌	❌	✅	🟡	✅	❌	🟡	✅	✅	❌	❌
GATED_DELTA_NET	❌	❌	✅	❌	🟡	❌	✅	🟡	✅	❌	❌
GATED_LINEAR_ATTN	❌	✅	✅	✅	❌	❌	✅	❌	❌	❌	❌
GEGLU	❌	✅	✅	✅	🟡	✅	✅	✅	✅	❌	❌
GEGLU_ERF	❌	✅	✅	✅	🟡	✅	✅	✅	✅	❌	❌
GEGLU_QUICK	❌	✅	✅	✅	🟡	✅	✅	✅	✅	❌	❌
GELU	❌	✅	✅	🟡	✅	🟡	✅	✅	✅	❌	❌
GELU_ERF	❌	✅	✅	🟡	✅	🟡	✅	✅	✅	❌	❌
GELU_QUICK	❌	✅	✅	🟡	✅	🟡	✅	✅	✅	❌	❌
GET_ROWS	❌	🟡	✅	🟡	🟡	🟡	✅	✅	🟡	❌	❌
GET_ROWS_BACK	❌	❌	🟡	🟡	❌	❌	❌	❌	❌	❌	❌
GROUP_NORM	❌	✅	✅	✅	✅	✅	✅	✅	❌	❌	❌
HARDSIGMOID	❌	✅	✅	🟡	✅	❌	✅	✅	✅	❌	❌
HARDSWISH	❌	✅	✅	🟡	✅	❌	✅	✅	✅	❌	❌
IM2COL	❌	✅	✅	✅	✅	✅	✅	✅	✅	❌	❌
IM2COL_3D	❌	❌	✅	✅	❌	❌	✅	✅	❌	❌	❌
L2_NORM	❌	✅	✅	✅	✅	❌	✅	✅	✅	❌	❌
LEAKY_RELU	❌	✅	✅	✅	🟡	❌	✅	🟡	❌	❌	❌
LOG	❌	✅	✅	✅	✅	❌	🟡	✅	✅	❌	❌
MEAN	❌	✅	✅	✅	✅	✅	✅	✅	❌	❌	❌
MUL	❌	✅	✅	✅	🟡	✅	✅	✅	✅	❌	❌
MUL_MAT	🟡	🟡	🟡	🟡	🟡	🟡	🟡	🟡	🟡	🟡	🟡
MUL_MAT_HADAMARD	❌	❌	❌	❌	❌	❌	✅	✅	❌	❌	❌
MUL_MAT_ID	❌	🟡	✅	✅	🟡	🟡	🟡	✅	🟡	🟡	❌
NEG	❌	✅	✅	🟡	✅	❌	✅	✅	✅	❌	❌
NORM	❌	✅	✅	✅	✅	✅	✅	🟡	✅	❌	❌
OPT_STEP_ADAMW	❌	❌	✅	✅	✅	❌	❌	✅	❌	❌	❌
OPT_STEP_SGD	❌	❌	✅	✅	✅	❌	❌	✅	❌	❌	❌
OUT_PROD	🟡	🟡	🟡	🟡	❌	❌	🟡	❌	❌	❌	🟡
PAD	❌	🟡	✅	🟡	🟡	🟡	🟡	✅	✅	❌	❌
PAD_REFLECT_1D	❌	✅	✅	✅	✅	❌	✅	❌	❌	❌	❌
POOL_1D	❌	❌	✅	❌	✅	❌	❌	❌	❌	❌	❌
POOL_2D	❌	🟡	✅	✅	✅	❌	✅	✅	❌	❌	❌
REGLU	❌	✅	✅	✅	🟡	✅	✅	✅	✅	❌	❌
RELU	❌	✅	✅	🟡	✅	🟡	✅	✅	✅	❌	❌
REPEAT	❌	✅	✅	🟡	✅	🟡	✅	✅	✅	❌	❌
REPEAT_BACK	❌	❌	✅	✅	❌	❌	✅	✅	❌	❌	❌
RMS_NORM	❌	✅	✅	✅	✅	✅	✅	✅	✅	❌	❌
RMS_NORM_BACK	❌	❌	✅	✅	❌	❌	✅	✅	❌	❌	❌
ROLL	❌	❌	✅	✅	✅	❌	✅	✅	❌	❌	❌
ROPE	❌	✅	✅	✅	✅	✅	✅	✅	✅	❌	❌
ROPE_BACK	❌	❌	✅	✅	❌	❌	✅	✅	❌	❌	❌
ROUND	❌	❌	✅	🟡	✅	❌	🟡	✅	✅	❌	❌
RWKV_WKV6	❌	❌	✅	✅	✅	❌	✅	✅	❌	❌	❌
RWKV_WKV7	❌	❌	✅	✅	✅	❌	✅	✅	❌	❌	❌
SCALE	❌	🟡	✅	✅	✅	✅	✅	✅	✅	❌	❌
SET	❌	❌	✅	✅	✅	❌	🟡	✅	✅	❌	❌
SET_ROWS	❌	🟡	🟡	🟡	🟡	🟡	🟡	🟡	🟡	❌	❌
SGN	❌	✅	✅	🟡	✅	❌	✅	✅	✅	❌	❌
SIGMOID	❌	✅	✅	🟡	✅	🟡	✅	✅	✅	❌	❌
SILU	❌	✅	✅	🟡	✅	🟡	✅	✅	✅	❌	❌
SILU_BACK	❌	❌	✅	✅	❌	❌	❌	✅	❌	❌	❌
SIN	❌	✅	✅	✅	✅	❌	🟡	🟡	✅	❌	❌
SOFTPLUS	❌	❌	✅	🟡	✅	❌	✅	✅	✅	❌	❌
SOFT_MAX	❌	🟡	✅	✅	✅	✅	✅	✅	✅	❌	❌
SOFT_MAX_BACK	❌	❌	🟡	🟡	❌	❌	🟡	✅	❌	❌	❌
SOLVE_TRI	❌	❌	✅	🟡	✅	❌	🟡	✅	✅	❌	❌
SQR	❌	✅	✅	✅	✅	✅	🟡	🟡	✅	❌	❌
SQRT	❌	✅	✅	✅	✅	✅	🟡	🟡	✅	❌	❌
SSM_CONV	❌	✅	✅	✅	✅	✅	✅	✅	✅	❌	❌
SSM_SCAN	❌	❌	✅	✅	✅	❌	🟡	🟡	✅	❌	❌
STEP	❌	✅	✅	🟡	✅	❌	✅	✅	✅	❌	❌
SUB	❌	✅	✅	✅	🟡	✅	✅	✅	✅	❌	❌
SUM	❌	🟡	✅	🟡	🟡	❌	🟡	🟡	🟡	❌	❌
SUM_ROWS	❌	✅	✅	🟡	✅	🟡	🟡	✅	✅	❌	❌
SWIGLU	❌	✅	✅	✅	🟡	✅	✅	✅	✅	❌	❌
SWIGLU_OAI	❌	❌	✅	✅	✅	✅	✅	✅	✅	❌	❌
TANH	❌	✅	✅	🟡	✅	✅	✅	✅	✅	❌	❌
TIMESTEP_EMBEDDING	❌	✅	✅	✅	✅	✅	✅	✅	❌	❌	❌
TOP_K	❌	❌	✅	❌	✅	❌	🟡	🟡	✅	❌	❌
TRI	❌	❌	✅	✅	✅	❌	✅	✅	✅	❌	❌
TRUNC	❌	❌	✅	🟡	✅	❌	🟡	✅	✅	❌	❌
UPSCALE	❌	🟡	✅	✅	✅	🟡	✅	✅	✅	❌	❌
XIELU	❌	❌	✅	❌	✅	❌	❌	✅	✅	❌	❌

11 KiB Raw Permalink Blame History

GGML Operations

How to add a backend to this table:

11 KiB

Raw Permalink Blame History