]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
overview / pkg / ggml / sources / ggml / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 82802be)
vulkan: Use unclamped loads for flash attention mask (llama/12720)
authorJeff Bolz <redacted>
Sun, 6 Apr 2025 08:47:13 +0000 (03:47 -0500)
committerGeorgi Gerganov <redacted>
Tue, 8 Apr 2025 08:47:46 +0000 (11:47 +0300)
commitfe9f9730dc9b8514503ea3796601528e6e047fc5
tree25a8fd266ef0cee3679bef269eed5d92e6624490tree
parent82802be175505df577fec5f81ac0bf4f256ff3a9commit | diff
vulkan: Use unclamped loads for flash attention mask (llama/12720)

nem1 must be a multiple of GGML_KQ_MASK_PAD, and GGML_KQ_MASK_PAD is a multiple
of the number of rows in the matrix. The KV dim is a multiple of the number of
columns for the aligned shader.
src/ggml-vulkan/ggml-vulkan.cpp diff | blob | history
src/ggml-vulkan/vulkan-shaders/flash_attn_cm2.comp diff | blob | history
Packaging of ggml-org/ggml