]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
overview / pkg / ggml / sources / ggml / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 5e0d451)
vulkan: add specific MMV kernels for IQ2 and IQ3 quants + optimizations (llama/11595)
authorRémy O <redacted>
Fri, 28 Feb 2025 08:42:52 +0000 (09:42 +0100)
committerGeorgi Gerganov <redacted>
Tue, 4 Mar 2025 19:24:42 +0000 (21:24 +0200)
commitacd91689647096c599e211f5061d55399db9290d
tree97596183e35e977a5d18c7c5f5e659a959dd3c93tree
parent5e0d4516f8bd548c509fb670fdb177fc1469e702commit | diff
vulkan: add specific MMV kernels for IQ2 and IQ3 quants + optimizations (llama/11595)

* vulkan: implement specialized MMV kernels for IQ2 quantizations

* vulkan: add MMV kernels for IQ3 quants

* vulkan: Increase MMV batch size and unroll IQ LUT setup

* vulkan: fix init_iq_shmem for WG sizes larger than tables

* vulkan: common batch size for all I-quants
src/ggml-vulkan/ggml-vulkan.cpp diff | blob | history
src/ggml-vulkan/vulkan-shaders/get_rows_quant.comp diff | blob | history
src/ggml-vulkan/vulkan-shaders/mul_mat_vec_iq2_s.comp [new file with mode: 0644] blob
src/ggml-vulkan/vulkan-shaders/mul_mat_vec_iq2_xs.comp [new file with mode: 0644] blob
src/ggml-vulkan/vulkan-shaders/mul_mat_vec_iq2_xxs.comp [new file with mode: 0644] blob
src/ggml-vulkan/vulkan-shaders/mul_mat_vec_iq3_s.comp [new file with mode: 0644] blob
src/ggml-vulkan/vulkan-shaders/mul_mat_vec_iq3_xxs.comp [new file with mode: 0644] blob
src/ggml-vulkan/vulkan-shaders/types.comp diff | blob | history
src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp diff | blob | history
Packaging of ggml-org/ggml