]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
sycl: fuse mul_mat(gate) + mul_mat(up) + GLU for q4_K dense FFN (llama/26779)
authorTitaniumtown <redacted>
Fri, 14 Aug 2026 06:26:23 +0000 (23:26 -0700)
committerGeorgi Gerganov <redacted>
Fri, 14 Aug 2026 19:16:06 +0000 (22:16 +0300)
commit2aef2a0085f115d45c2a4adf5e706c42f070c854
tree0e3f695b005c863b676c099601f9b29bf1d74a23
parent5bc0c8525445045d35e013dcc26d57d20f6283e6
sycl: fuse mul_mat(gate) + mul_mat(up) + GLU for q4_K dense FFN (llama/26779)

Measured on Arc Pro B70 (Battlemage, Level Zero), llama-bench -r 20, two
interleaved rounds, tg128:

    qwen2.5-3B-Instruct Q4_K_M    154.18 -> 158.53 t/s   +2.8%
    gemma-2-2b-it Q4_K_M          162.45 -> 165.62 t/s   +2.0%

llama-batched-bench on qwen2.5-3B, S_TG by batch size:

      B=1   142.72 -> 147.57 t/s    +3.4%
      B=2   243.72 -> 268.26 t/s   +10.1%
      B=4   359.58 -> 398.02 t/s   +10.7%
      B=8   449.75 -> 505.63 t/s   +12.4%
ggml/src/ggml-sycl/element_wise.cpp
ggml/src/ggml-sycl/element_wise.hpp
ggml/src/ggml-sycl/fusion.cpp
ggml/src/ggml-sycl/ggml-sycl.cpp
ggml/src/ggml-sycl/mmvq.cpp
ggml/src/ggml-sycl/mmvq.hpp