]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
overview / pkg / ggml / sources / llama.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: e251e5e)
vulkan: Reduce temporary memory usage for TOP_K (#17623)
authorJeff Bolz <redacted>
Tue, 2 Dec 2025 18:22:04 +0000 (12:22 -0600)
committerGitHub <redacted>
Tue, 2 Dec 2025 18:22:04 +0000 (19:22 +0100)
commit61bde8e21f4a1f9a98c9205831ca3e55457b4c78
tree46cfa4db276c62978475e761201878ff5228bf63tree
parente251e5ebbee9c769ecb746377f9c5e12c0a0a0d2commit | diff
vulkan: Reduce temporary memory usage for TOP_K (#17623)

- Compute row size for the temp buffer based on the output of the first pass.
- Update shader addressing math to use the output row size
- Pass the output row size as "ncols_output", what used to be "ncols_output" is now "k"

For the common case of K=40 and src0=(200000,1,1,1), this reduces the temporary buffer
from about 3.2MB to 500KB.
ggml/src/ggml-vulkan/ggml-vulkan.cpp diff | blob | history
ggml/src/ggml-vulkan/vulkan-shaders/topk_argsort.comp diff | blob | history
ggml/src/ggml-vulkan/vulkan-shaders/topk_nary_search.comp diff | blob | history
Packaging of ggml-org/llama.cpp