]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
vulkan: Use larger workgroups for mul_mat_vec when M is small (llama/15355)
authorJeff Bolz <redacted>
Sun, 17 Aug 2025 16:08:57 +0000 (11:08 -0500)
committerGeorgi Gerganov <redacted>
Mon, 18 Aug 2025 16:15:25 +0000 (19:15 +0300)
commit79c0d779d1c2de0adabe6bca96f2a5fd3667db57
tree12d5a9d1ab365b55d4c932cb9f4bb776188e4377
parent403a8ea7c149ec58bfb56cba9e8651dbdce03767
vulkan: Use larger workgroups for mul_mat_vec when M is small (llama/15355)

* vulkan: Use larger workgroups for mul_mat_vec when M is small

Also use subgroup instructions for (part of) the reduction when supported.
Without this, the more expensive reductions would eat into the benefits of
the larger workgroups.

* update heuristic for amd/intel

Co-authored-by: 0cc4m <redacted>
---------

Co-authored-by: 0cc4m <redacted>
src/ggml-vulkan/ggml-vulkan.cpp
src/ggml-vulkan/vulkan-shaders/mul_mat_vec_base.comp
src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp