]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
vulkan: multi-row k quants (llama/10846)
authorEve <redacted>
Thu, 26 Dec 2024 15:54:44 +0000 (10:54 -0500)
committerGeorgi Gerganov <redacted>
Fri, 3 Jan 2025 12:00:38 +0000 (14:00 +0200)
commit9501d5c4540b7205f0992ef295ec42be930c74cd
tree9dcf9d1023eb2a0c59141fac1a6ce81543c1b871
parentc8779fa3dc5db9c81d484c469f41d4a053072f2e
vulkan: multi-row k quants (llama/10846)

* multi row k quant shaders!

* better row selection

* more row choices

* readjust row selection

* rm_kq=2 by default
src/ggml-vulkan/ggml-vulkan.cpp
src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q2_k.comp
src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q3_k.comp
src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q4_k.comp
src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q5_k.comp
src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q6_k.comp