]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
overview / pkg / ggml / sources / llama.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 7cc2d2c)
vulkan: Dynamic subgroup size support for Q6_K mat_vec (#10536)
authorEve <redacted>
Sat, 30 Nov 2024 07:00:02 +0000 (07:00 +0000)
committerGitHub <redacted>
Sat, 30 Nov 2024 07:00:02 +0000 (08:00 +0100)
commit0533e7fb3842a523f64dc533bd7bd7147ec2c63a
tree24d489e1ff140dcfb02d76f4ecccb766bde42fe7tree
parent7cc2d2c88908fc92b97b28acafb82f7d6e425b85commit | diff
vulkan: Dynamic subgroup size support for Q6_K mat_vec (#10536)

* subgroup 64 version with subgroup add. 15% faster

scalable version

tested for subgroup sizes 16-128

* check for subgroup multiple of 16 and greater than 16

* subgroup sizes are always a power of 2 (https://github.com/KhronosGroup/GLSL/issues/45)

* force 16 sequential threads per block

* make 16 subgroup size a constant
ggml/src/ggml-vulkan/ggml-vulkan.cpp diff | blob | history
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q6_k.comp diff | blob | history
Packaging of ggml-org/llama.cpp