]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
overview / pkg / ggml / sources / ggml / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 66c72fd)
vulkan: Hybrid waitForFences/getFenceStatus to reduce fence latency (llama/12630)
authorJeff Bolz <redacted>
Fri, 4 Apr 2025 05:54:35 +0000 (00:54 -0500)
committerGeorgi Gerganov <redacted>
Tue, 8 Apr 2025 08:47:46 +0000 (11:47 +0300)
commit0cedece5e2a067f89aad9e0261779f1073d0a700
tree0226c954a921ba2e60f83da2c8728f9331747eadtree
parent66c72fd36bf4656d42f884d282ebada47e4dbefecommit | diff
vulkan: Hybrid waitForFences/getFenceStatus to reduce fence latency (llama/12630)

There seems to be a bubble waking up from waitForFences, which costs a few
percent performance and also increased variance in performance. This change
inserts an "almost_ready" fence when the graph is about 80% complete and we
waitForFences for the almost_ready fence and then spin (with _mm_pauses) waiting
for the final fence to be signaled.
src/ggml-vulkan/ggml-vulkan.cpp diff | blob | history
Packaging of ggml-org/ggml