]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
overview / pkg / ggml / sources / ggml / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 40376b7)
vulkan: optimize flash attention split_k_reduce (llama/14554)
authorJeff Bolz <redacted>
Tue, 8 Jul 2025 18:11:42 +0000 (13:11 -0500)
committerGeorgi Gerganov <redacted>
Sat, 12 Jul 2025 13:05:00 +0000 (16:05 +0300)
commit2f52a6fa90c10d414820624c5b5aa765ce460444
treec561eda63710427e78e5c916f4ef9a915070e592tree
parent40376b718215388edb99d517cd5467f75e7205aecommit | diff
vulkan: optimize flash attention split_k_reduce (llama/14554)

* vulkan: allow FA split_k with smaller KV values

* vulkan: spread split_k_reduce work across more threads

k_num can get rather large. Use the whole workgroup to reduce the M/L values.

Launch a thread for each element in the HSV dimension of the output. Helps a
lot for large HSV (like deepseek).
src/ggml-vulkan/ggml-vulkan.cpp diff | blob | history
src/ggml-vulkan/vulkan-shaders/flash_attn_split_k_reduce.comp diff | blob | history
Packaging of ggml-org/ggml