]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
overview / pkg / ggml / sources / llama.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 699f439)
vulkan: optimize flash attention split_k_reduce (#14554)
authorJeff Bolz <redacted>
Tue, 8 Jul 2025 18:11:42 +0000 (13:11 -0500)
committerGitHub <redacted>
Tue, 8 Jul 2025 18:11:42 +0000 (20:11 +0200)
commit6efcd65945a98cf6883cdd9de4c8ccd8c79d219a
treef00aafb7db6a88619849da6aae264d23f011fb16tree
parent699f4392a33f57c3352cf8d60bdc53db7ca235e7commit | diff
vulkan: optimize flash attention split_k_reduce (#14554)

* vulkan: allow FA split_k with smaller KV values

* vulkan: spread split_k_reduce work across more threads

k_num can get rather large. Use the whole workgroup to reduce the M/L values.

Launch a thread for each element in the HSV dimension of the output. Helps a
lot for large HSV (like deepseek).
ggml/src/ggml-vulkan/ggml-vulkan.cpp diff | blob | history
ggml/src/ggml-vulkan/vulkan-shaders/flash_attn_split_k_reduce.comp diff | blob | history
Packaging of ggml-org/llama.cpp