]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
vulkan: Reduce temporary memory usage for TOP_K (llama/17623)
authorJeff Bolz <redacted>
Tue, 2 Dec 2025 18:22:04 +0000 (12:22 -0600)
committerGeorgi Gerganov <redacted>
Fri, 12 Dec 2025 15:53:15 +0000 (17:53 +0200)
commit86cb5ab93fcbf2a5de6ed68569be8a1cb85562f9
tree7992d89c51cfb3330a96f5d448d922d0101c9f62
parentfffdf679d4930520a7be5f4e794666ddab498981
vulkan: Reduce temporary memory usage for TOP_K (llama/17623)

- Compute row size for the temp buffer based on the output of the first pass.
- Update shader addressing math to use the output row size
- Pass the output row size as "ncols_output", what used to be "ncols_output" is now "k"

For the common case of K=40 and src0=(200000,1,1,1), this reduces the temporary buffer
from about 3.2MB to 500KB.
ggml/src/ggml-vulkan/ggml-vulkan.cpp
ggml/src/ggml-vulkan/vulkan-shaders/topk_argsort.comp
ggml/src/ggml-vulkan/vulkan-shaders/topk_nary_search.comp