]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
vulkan: multi-row k quants (#10846)
authorEve <redacted>
Thu, 26 Dec 2024 15:54:44 +0000 (10:54 -0500)
committerGitHub <redacted>
Thu, 26 Dec 2024 15:54:44 +0000 (16:54 +0100)
commitd79d8f39b4da6deca4aea8bf130c6034c482b320
tree12db0095634d79cebd31615059bd488f70f985c7
parentd283d02bf254a7f2991e1502066330cc0d4321a6
vulkan: multi-row k quants (#10846)

* multi row k quant shaders!

* better row selection

* more row choices

* readjust row selection

* rm_kq=2 by default
ggml/src/ggml-vulkan/ggml-vulkan.cpp
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q2_k.comp
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q3_k.comp
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q4_k.comp
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q5_k.comp
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q6_k.comp