]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
sycl: parallelize the non-contiguous concat kernel (llama/25852)
authorTitaniumtown <redacted>
Tue, 4 Aug 2026 02:08:05 +0000 (19:08 -0700)
committerGeorgi Gerganov <redacted>
Tue, 4 Aug 2026 10:37:47 +0000 (13:37 +0300)
commit6217ef7468443defc02d2adaec01d7cb82ae4686
tree94b96500d377cf05bc22f6ecaa1942ad7cbb2670
parent97abf5c753ed432358deb37bb15a9e1e0bf0179c
sycl: parallelize the non-contiguous concat kernel (llama/25852)

* sycl: parallelize the non-contiguous concat kernel

Launch geometry only: the non-contiguous concat kernel launched a single-lane
work-group (1, 1, 1), now it will launch a (1, 1, SYCL_CONCAT_BLOCK_SIZE) one.

SYCL_CONCAT_BLOCK_SIZE is defined in `ggml/src/ggml-sycl/presets.hpp`.

llama-bench (Arc Pro B70, Qwen3.6-27B-UD-Q4_K_XL, -fa on, q8_0 KV),
on top of upstream master: pp2048 920 -> 1006 t/s (+9.4%)

* sycl: cap non-contiguous concat block at ne0

* sycl: make non-contiguous concat block width env-tunable (GGML_SYCL_CONCAT_BLOCK_SIZE)

* Revert "sycl: make non-contiguous concat block width env-tunable (GGML_SYCL_CONCAT_BLOCK_SIZE)"

This reverts commit 2709909e790d298cc0746950caff16ad8ecf9d6f.
ggml/src/ggml-sycl/concat.cpp