]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
vulkan: support SET_ROWS (llama/14587)
authorJeff Bolz <redacted>
Sat, 12 Jul 2025 10:12:26 +0000 (05:12 -0500)
committerGeorgi Gerganov <redacted>
Sat, 12 Jul 2025 16:23:56 +0000 (19:23 +0300)
commit915fc153a528d4f9a1c8616658aa042f0366febb
tree81244d48380214df1cfb53cb880208c32072f4aa
parent8670a3fd5def4cbcb5742bbc862c609acbc9c061
vulkan: support SET_ROWS (llama/14587)

* vulkan: support SET_ROWS

Add variants of the copy_to_quant shader that do the SET_ROWS operation.
Change these shaders to spread the work across the workgroup.
The memory access pattern is probably not great (one thread per quant block),
but should be fine for now.

* vulkan: optimize set_rows

Larger workgroups for non-quant types.
Set "norepeat" (there is manual repeat logic).
Use fastmod.
ggml/src/ggml-vulkan/ggml-vulkan.cpp
ggml/src/ggml-vulkan/vulkan-shaders/copy_to_quant.comp
ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp