]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
sycl : set K_QUANTS_PER_ITERATION to 1 on DMMV path (llama/25063)
authorTodd Malsbary <redacted>
Tue, 7 Jul 2026 07:43:41 +0000 (00:43 -0700)
committerGeorgi Gerganov <redacted>
Fri, 10 Jul 2026 10:06:42 +0000 (13:06 +0300)
commite1d865d8340aefeeb47877bd5cc485c43dd6982e
treef0803f7a9265363263283ba265b210da710369b4
parent120fc3729de83015e75198c171020c5d08ae9695
sycl : set K_QUANTS_PER_ITERATION to 1 on DMMV path (llama/25063)

* sycl: add supported types to ggml_sycl_supports_reorder_dmmv

The reordered feature is implemented in ggml_sycl_op_dequantize_mul_mat_vec,
but gated by ggml_sycl_supports_reorder_dmmv. This commit fixes the gate.

Signed-off-by: Todd Malsbary <redacted>
* sycl: set K_QUANTS_PER_ITERATION=1 to improve utilization

When combined with opening the reorder gate, this improves GPU
utilization on B70, giving a significant boost to tg t/s.

Signed-off-by: Todd Malsbary <redacted>
* sycl: replace QK_WARP_SIZE with WARP_SIZE for QK_5

Signed-off-by: Todd Malsbary <redacted>
* sycl: add missing types to ggml_backend_sycl_buffer_init_tensor

Without this, the extra field is not allocated and the reorder path
will not take effect.

Signed-off-by: Todd Malsbary <redacted>
---------

Signed-off-by: Todd Malsbary <redacted>
ggml/src/ggml-sycl/dmmv.cpp
ggml/src/ggml-sycl/ggml-sycl.cpp
ggml/src/ggml-sycl/presets.hpp