]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
vulkan: Allow up to 4096 elements for mul_mat_id row_ids (llama/13326)
authorJeff Bolz <redacted>
Fri, 9 May 2025 07:23:41 +0000 (02:23 -0500)
committerGeorgi Gerganov <redacted>
Tue, 13 May 2025 10:02:19 +0000 (13:02 +0300)
commit3f516b945b43ab425ca4c8db33946990bdbddab0
tree5da044cce977f3917fb12389aed9d02e65e904f0
parenta90f46badf7f85fa96048084f6bc75e281f0bcbf
vulkan: Allow up to 4096 elements for mul_mat_id row_ids (llama/13326)

This assert fired running Qwen_Qwen3-30B-A3B-Q2_K.gguf:

GGML_ASSERT(nei0 * nei1 <= 3072);

The tensor is 8 x 512. Increase this array size to accommodate.
src/ggml-vulkan/ggml-vulkan.cpp
src/ggml-vulkan/vulkan-shaders/mul_mm.comp
src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp
src/ggml-vulkan/vulkan-shaders/mul_mmq.comp