]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
vulkan: support arbitrary KV dimension in flash attention (llama/16160)
authorJeff Bolz <redacted>
Sat, 27 Sep 2025 20:43:39 +0000 (16:43 -0400)
committerGeorgi Gerganov <redacted>
Mon, 29 Sep 2025 09:41:09 +0000 (12:41 +0300)
commit2549fa3c25192af374fe3a771f16847539911ff2
treef93e5aeac10d1c4b7323a00bcc1caf066265cbfb
parent503ed49d03af353fa5fce8486f24bb1e455187b5
vulkan: support arbitrary KV dimension in flash attention (llama/16160)

The "Clamp" spec constant is already based on whether KV is a multiple of Bc,
so use that to control whether bounds checking is performed. Add bounds checking
to the scalar and coopmat1 paths. Coopmat2 didn't need any changes (the K/V
tensors are already optionally clamped, nothing else needed to be changed).
src/ggml-vulkan/vulkan-shaders/flash_attn.comp
src/ggml-vulkan/vulkan-shaders/flash_attn_base.comp
src/ggml-vulkan/vulkan-shaders/flash_attn_cm1.comp