]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
overview / pkg / ggml / sources / llama.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 6ab4e50)
vulkan: Implement top-k (#17418)
authorJeff Bolz <redacted>
Wed, 26 Nov 2025 15:45:43 +0000 (09:45 -0600)
committerGitHub <redacted>
Wed, 26 Nov 2025 15:45:43 +0000 (16:45 +0100)
commit879d673759181987300a29989478c0f36c97e97b
treece87a63cc1123e960beed05b06c66cc2811bfa04tree
parent6ab4e50d9c65e7d94ab7ce135114b52e484019abcommit | diff
vulkan: Implement top-k (#17418)

* vulkan: Implement top-k

Each pass launches workgroups that each sort 2^N elements (where N is usually 7-10)
and discards all but the top K. Repeat until only K are left. And there's a fast
path when K==1 to just find the max value rather than sorting.

* fix pipeline selection

* vulkan: Add N-ary search algorithm for topk

* microoptimizations
ggml/src/ggml-vulkan/ggml-vulkan.cpp diff | blob | history
ggml/src/ggml-vulkan/vulkan-shaders/topk_argsort.comp [new file with mode: 0644] blob
ggml/src/ggml-vulkan/vulkan-shaders/topk_nary_search.comp [new file with mode: 0644] blob
ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp diff | blob | history
tests/test-backend-ops.cpp diff | blob | history
Packaging of ggml-org/llama.cpp