]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
sycl : set K_QUANTS_PER_ITERATION to 1 on DMMV path (#25063)
authorTodd Malsbary <redacted>
Tue, 7 Jul 2026 07:43:41 +0000 (00:43 -0700)
committerGitHub <redacted>
Tue, 7 Jul 2026 07:43:41 +0000 (10:43 +0300)
commitd209086157f007ffa672991518634b8f283522b2
treee30c2631369474cb76253d1bd1f94824eb610134
parent95e5254c0ae93529b2f6c05e210b9cac5c0070fc
sycl : set K_QUANTS_PER_ITERATION to 1 on DMMV path (#25063)

* sycl: add supported types to ggml_sycl_supports_reorder_dmmv

The reordered feature is implemented in ggml_sycl_op_dequantize_mul_mat_vec,
but gated by ggml_sycl_supports_reorder_dmmv. This commit fixes the gate.

Signed-off-by: Todd Malsbary <redacted>
* sycl: set K_QUANTS_PER_ITERATION=1 to improve utilization

When combined with opening the reorder gate, this improves GPU
utilization on B70, giving a significant boost to tg t/s.

Signed-off-by: Todd Malsbary <redacted>
* sycl: replace QK_WARP_SIZE with WARP_SIZE for QK_5

Signed-off-by: Todd Malsbary <redacted>
* sycl: add missing types to ggml_backend_sycl_buffer_init_tensor

Without this, the extra field is not allocated and the reorder path
will not take effect.

Signed-off-by: Todd Malsbary <redacted>
---------

Signed-off-by: Todd Malsbary <redacted>
ggml/src/ggml-sycl/dmmv.cpp
ggml/src/ggml-sycl/ggml-sycl.cpp
ggml/src/ggml-sycl/presets.hpp