]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
vulkan: enable coopmat2 FA gqa and split_k optimizations more often (llama/12931)
authorJeff Bolz <redacted>
Wed, 16 Apr 2025 18:37:25 +0000 (13:37 -0500)
committerGeorgi Gerganov <redacted>
Thu, 24 Apr 2025 17:39:16 +0000 (20:39 +0300)
commit7db8f278f0fe59c42fba40f9f3cf8abe17019b89
tree05bfdcf6b0c2184f0557cd2ab375cb502a6487a8
parentbe42a19eab9d3c07b46348fe3a0154e8d6ff9a32
vulkan: enable coopmat2 FA gqa and split_k optimizations more often (llama/12931)

The grouped query attention optmization doesn't require a power of two ratio,
the only thing relying on it was the modulo operation written as bitwise &.

split_k need not depend on gqa_ratio - enable it any time there's only one
workgroup in the X dimension. The shader gets the split index from the x coord,
and multiple workgroups in the X dimension (pre-split) indicates a larger
FA operation that wouldn't need splitting.
ggml/src/ggml-vulkan/ggml-vulkan.cpp
ggml/src/ggml-vulkan/vulkan-shaders/flash_attn_cm2.comp