]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
overview / pkg / ggml / sources / llama.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: a057897)
server : fix cache reuse logic (#12161)
authorClauszy <redacted>
Wed, 5 Mar 2025 07:25:45 +0000 (15:25 +0800)
committerGitHub <redacted>
Wed, 5 Mar 2025 07:25:45 +0000 (09:25 +0200)
commit06a92a193a07afe445929607be9d5e4d033956fb
treef00d35056b1eb2e408c289a235e733aaf5a07989tree
parenta057897ad4e48e3df0354256e0cc80dadcb57595commit | diff
server : fix cache reuse logic (#12161)

The first kv shift offsets the positions of all tokens after head_c.
When using llama_kv_cache_seq_rm next, using head_c will remove the valid tokens because their positions have already been offset.
examples/server/server.cpp diff | blob | history
Packaging of ggml-org/llama.cpp