]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
overview / pkg / ggml / sources / llama.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: abbabc5)
server : fix crash when system prompt is bigger than batch size (#5714)
authorcompilade <redacted>
Sun, 25 Feb 2024 18:43:50 +0000 (13:43 -0500)
committerGitHub <redacted>
Sun, 25 Feb 2024 18:43:50 +0000 (20:43 +0200)
commitf7625019c51ca437a5840576d92362cfa710e4a2
tree6bfc6ccfd3f00857759192a1458a31f1d0b755d9tree
parentabbabc5e51d0d4656b438aec10b7fae9479ef37dcommit | diff
server : fix crash when system prompt is bigger than batch size (#5714)

The system prompt is now decoded in batches.

* server : fix off-by-one n_past when start of prompt matches whole cache

The tokens right after the matching part would otherwise skip a pos value.
examples/server/server.cpp diff | blob | history
Packaging of ggml-org/llama.cpp