]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
overview / pkg / ggml / sources / llama.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 0f7fada)
vulkan: Use cm2 decode_vector for mul_mat_id B matrix loads (#23991)
authorJeff Bolz <redacted>
Mon, 8 Jun 2026 08:40:37 +0000 (03:40 -0500)
committerGitHub <redacted>
Mon, 8 Jun 2026 08:40:37 +0000 (10:40 +0200)
commitc74759a244df27e460e20fe6248c6efb9249e62f
treeb775956953e7d969ce1bf03d087f289cd9f9b07btree
parent0f7fada56bea32c9e0db3874b801d887ff6c4529commit | diff
vulkan: Use cm2 decode_vector for mul_mat_id B matrix loads (#23991)

This allows vec4 loads of the B elements. Also increase BK to 64 when this is
enabled. Neither of these alone is consistently faster, but together these give
a nice speedup.

In ggml-vulkan.cpp, we need to make sure the B matrix alignment and stride are
multiples of 4.
ggml/src/ggml-vulkan/ggml-vulkan.cpp diff | blob | history
ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp diff | blob | history
ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp diff | blob | history
tests/test-backend-ops.cpp diff | blob | history
Packaging of ggml-org/llama.cpp