]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
sycl: fuse mul_mat(gate) + mul_mat(up) + GLU for q4_K dense FFN (#26779)
authorTitaniumtown <redacted>
Fri, 14 Aug 2026 06:26:23 +0000 (23:26 -0700)
committerGitHub <redacted>
Fri, 14 Aug 2026 06:26:23 +0000 (02:26 -0400)
commit65091386227039bfb81ee3426537656e3b4a3f83
tree620d19bef518fe90fff54fb46e8d935255296a37
parentc6f6a92c553984a7f59b712d3a1694fed275e6c3
sycl: fuse mul_mat(gate) + mul_mat(up) + GLU for q4_K dense FFN (#26779)

Measured on Arc Pro B70 (Battlemage, Level Zero), llama-bench -r 20, two
interleaved rounds, tg128:

    qwen2.5-3B-Instruct Q4_K_M    154.18 -> 158.53 t/s   +2.8%
    gemma-2-2b-it Q4_K_M          162.45 -> 165.62 t/s   +2.0%

llama-batched-bench on qwen2.5-3B, S_TG by batch size:

      B=1   142.72 -> 147.57 t/s    +3.4%
      B=2   243.72 -> 268.26 t/s   +10.1%
      B=4   359.58 -> 398.02 t/s   +10.7%
      B=8   449.75 -> 505.63 t/s   +12.4%
docs/backend/SYCL.md
ggml/src/ggml-sycl/element_wise.cpp
ggml/src/ggml-sycl/element_wise.hpp
ggml/src/ggml-sycl/fusion.cpp
ggml/src/ggml-sycl/ggml-sycl.cpp
ggml/src/ggml-sycl/mmvq.cpp
ggml/src/ggml-sycl/mmvq.hpp
tests/test-backend-ops.cpp