]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
overview / pkg / ggml / sources / llama.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 344f912)
Avoid unnecessarily disabling CUDA graphs (#7302)
authoragray3 <redacted>
Wed, 15 May 2024 13:44:49 +0000 (14:44 +0100)
committerGitHub <redacted>
Wed, 15 May 2024 13:44:49 +0000 (15:44 +0200)
commitdc020985b8755dd6aa93a2f002f43c3ede808cce
treea4be81a8ce9f08fbafbc92c3e38ee892192bfe91tree
parent344f9126cc0d15891fde9472fe40b8572628ad7dcommit | diff
Avoid unnecessarily disabling CUDA graphs (#7302)

As discussed in PR #6766, CUDA graphs were being disabled in the presence of long prompts.
This fixes the issue by avoiding the consective update counter from incrementing unnecessarily
for tokens in which cuda graphs are disabled due to batch size > 1.
ggml-cuda.cu diff | blob | history
Packaging of ggml-org/llama.cpp