mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2026-07-08 12:10:25 +00:00
Todd Malsbary
d209086157
* sycl: add supported types to ggml_sycl_supports_reorder_dmmv The reordered feature is implemented in ggml_sycl_op_dequantize_mul_mat_vec, but gated by ggml_sycl_supports_reorder_dmmv. This commit fixes the gate. Signed-off-by: Todd Malsbary <todd.malsbary@intel.com> * sycl: set K_QUANTS_PER_ITERATION=1 to improve utilization When combined with opening the reorder gate, this improves GPU utilization on B70, giving a significant boost to tg t/s. Signed-off-by: Todd Malsbary <todd.malsbary@intel.com> * sycl: replace QK_WARP_SIZE with WARP_SIZE for QK_5 Signed-off-by: Todd Malsbary <todd.malsbary@intel.com> * sycl: add missing types to ggml_backend_sycl_buffer_init_tensor Without this, the extra field is not allocated and the reorder path will not take effect. Signed-off-by: Todd Malsbary <todd.malsbary@intel.com> --------- Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>