]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
vulkan: dynamic subgroup size for the remaining k quants (llama/10745)
authorEve <redacted>
Tue, 10 Dec 2024 19:33:23 +0000 (19:33 +0000)
committerGeorgi Gerganov <redacted>
Wed, 18 Dec 2024 10:52:16 +0000 (12:52 +0200)
commitd8bf63a41b934eed168a486904cb2e57e01dff38
tree7f4dbb85053bfcccd785f8ee8fb12de0950a15da
parentb82c8d76dc39f437703693123660ddd4f7dede1c
vulkan: dynamic subgroup size for the remaining k quants (llama/10745)

* q5_k

q4_k

q3_k

q2_k

q6_k multi row example

* revert as multi row isnt faster for k quants
ggml/src/ggml-vulkan/ggml-vulkan.cpp
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_base.comp
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q2_k.comp
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q3_k.comp
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q4_k.comp
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q5_k.comp