]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
overview / pkg / ggml / sources / whisper.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 3623bd5)
vulkan: Dynamic subgroup size support for Q6_K mat_vec (llama/10536)
authorEve <redacted>
Sat, 30 Nov 2024 07:00:02 +0000 (07:00 +0000)
committerGeorgi Gerganov <redacted>
Sun, 8 Dec 2024 18:14:35 +0000 (20:14 +0200)
commit30e35d7271ac6c87aa93cffb225f06b254ec4a7a
tree1da4a591852141212c7b46ed4618819a764b1f68tree
parent3623bd58f221dfd54f5667240ffd663df45010dbcommit | diff
vulkan: Dynamic subgroup size support for Q6_K mat_vec (llama/10536)

* subgroup 64 version with subgroup add. 15% faster

scalable version

tested for subgroup sizes 16-128

* check for subgroup multiple of 16 and greater than 16

* subgroup sizes are always a power of 2 (https://github.com/KhronosGroup/GLSL/issues/45)

* force 16 sequential threads per block

* make 16 subgroup size a constant
ggml/src/ggml-vulkan/ggml-vulkan.cpp diff | blob | history
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q6_k.comp diff | blob | history
Packaging of ggerganov/whisper.cpp