]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
Vulkan: MMVQ Integer Dot K-Quant and MUL_MAT_ID support (llama/16900)
authorRuben Ortlam <redacted>
Sat, 29 Nov 2025 08:37:22 +0000 (09:37 +0100)
committerGeorgi Gerganov <redacted>
Fri, 12 Dec 2025 15:53:12 +0000 (17:53 +0200)
commit2fcc0a3a9ff3a40a150e5dc09395bb0a985b7ad8
tree2a495510d3f295414d38f6c72f9e095c76b8ea14
parentdbf8766ffad8e36cdd4176a92d62f3d4dece6c1b
Vulkan: MMVQ Integer Dot K-Quant and MUL_MAT_ID support (llama/16900)

* vulkan: split mul_mmq_funcs for mul_mat_vecq use

* add mxfp4 mmvq

* add q2_k mmvq

* add q3_k mmvq

* add q4_k and q5_k mmvq

* add q6_k mmvq

* handle 4x4 quants per mmvq thread

* enable MUL_MAT_ID mmvq support

* enable subgroup optimizations for mul_mat_vec_id shaders

* device tuning

* request prealloc_y sync after quantization

* fix indentation

* fix llvmpipe test failures

* fix mul_mat_id mmvq condition

* fix unused variable warning
12 files changed:
ggml/src/ggml-vulkan/ggml-vulkan.cpp
ggml/src/ggml-vulkan/vulkan-shaders/dequant_funcs.glsl
ggml/src/ggml-vulkan/vulkan-shaders/generic_binary_head.glsl
ggml/src/ggml-vulkan/vulkan-shaders/generic_unary_head.glsl
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec.comp
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_base.glsl
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_iface.glsl
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vecq.comp
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vecq_funcs.glsl [new file with mode: 0644]
ggml/src/ggml-vulkan/vulkan-shaders/mul_mmq.comp
ggml/src/ggml-vulkan/vulkan-shaders/mul_mmq_funcs.glsl
ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp