]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
overview / pkg / ggml / sources / ggml / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 2549fa3)
vulkan: handle mat_mul with A matrix > 4GB (llama/16176)
authorJeff Bolz <redacted>
Sun, 28 Sep 2025 01:36:34 +0000 (20:36 -0500)
committerGeorgi Gerganov <redacted>
Mon, 29 Sep 2025 09:41:09 +0000 (12:41 +0300)
commitbf14b631b93a12ed4e0a905470d7e677ab9599de
tree01e2a2db244380a73eb2bc18d810f65a6a339a06tree
parent2549fa3c25192af374fe3a771f16847539911ff2commit | diff
vulkan: handle mat_mul with A matrix > 4GB (llama/16176)

* vulkan: handle mat_mul with A matrix > 4GB

This change splits mat_mul operations with huge A matrix into chunks in the M
dimension. This works well for stable-diffusion use cases where the im2col
matrix has very large M.

Fix the order of setting the stride in mul_mm_cm2 - setting the dimension
clobbers the stride, so stride should be set after.

* build fixes
src/ggml-vulkan/ggml-vulkan.cpp diff | blob | history
src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp diff | blob | history
tests/test-backend-ops.cpp diff | blob | history
Packaging of ggml-org/ggml