]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
vulkan: dynamic subgroup size for the remaining k quants (llama/10745)
authorEve <redacted>
Tue, 10 Dec 2024 19:33:23 +0000 (19:33 +0000)
committerGeorgi Gerganov <redacted>
Tue, 17 Dec 2024 17:23:40 +0000 (19:23 +0200)
commitff32488768685015f8be5f02e8387dc0629124ad
tree3ddc13f7ef5c4a812ca68020b3c4626394258b41
parent0ea6edab083d3ceba84ce56ecbb0e82ba739f9c5
vulkan: dynamic subgroup size for the remaining k quants (llama/10745)

* q5_k

q4_k

q3_k

q2_k

q6_k multi row example

* revert as multi row isnt faster for k quants
src/ggml-vulkan/ggml-vulkan.cpp
src/ggml-vulkan/vulkan-shaders/mul_mat_vec_base.comp
src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q2_k.comp
src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q3_k.comp
src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q4_k.comp
src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q5_k.comp