]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
vulkan: small mul_mat_vec optimizations (llama/10665)
authorEve <redacted>
Fri, 13 Dec 2024 08:42:04 +0000 (08:42 +0000)
committerGeorgi Gerganov <redacted>
Wed, 18 Dec 2024 10:52:16 +0000 (12:52 +0200)
commitc21fb10b284ce0d3ee116be96447a8cc8c7a0994
treea6e3c89c835bf908735052bdf7a443cc23fbac14
parent26c9fd0cdca703684b4e0c4bfaaf45b306c79f43
vulkan: small mul_mat_vec optimizations (llama/10665)

* double the number of rows per workgroup

* Update ggml-vulkan.cpp

* Vulkan: Add VK_EXT_subgroup_size_control support to ensure full subgroups for coopmats

* only increase the number of rows for amd and subgroup size 64

* fix missing NUM_ROWS for mul_mat_vec_iq4_nl_f16_f32, untested

* use subgroup min and max to check for gcn (requires https://github.com/ggerganov/llama.cpp/pull/10721)

* manual merge ggml-vulkan.cpp

* set min and max subgroup size in any case

* Also double the number of rows for Intel GPUs
ggml/src/ggml-vulkan/ggml-vulkan.cpp
ggml/src/ggml-vulkan/vulkan-shaders/dequant_funcs.comp
ggml/src/ggml-vulkan/vulkan-shaders/get_rows_quant.comp
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec.comp