]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
vulkan: multi-row k quants (llama/10846)
authorEve <redacted>
Thu, 26 Dec 2024 15:54:44 +0000 (10:54 -0500)
committerGeorgi Gerganov <redacted>
Sat, 4 Jan 2025 08:45:01 +0000 (10:45 +0200)
commit8de1e999078aa957cb4886d06cae23e378911297
tree6d1ab70dcebcf37d7678b8ded32ee13757b1ebab
parent499af9294a861883c2730f87fa7d32c1e46434ae
vulkan: multi-row k quants (llama/10846)

* multi row k quant shaders!

* better row selection

* more row choices

* readjust row selection

* rm_kq=2 by default
ggml/src/ggml-vulkan/ggml-vulkan.cpp
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q2_k.comp
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q3_k.comp
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q4_k.comp
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q5_k.comp
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_q6_k.comp