]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
server: fix checkpoints creation (#22929)
authorjacekpoplawski <redacted>
Mon, 25 May 2026 05:56:18 +0000 (07:56 +0200)
committerGitHub <redacted>
Mon, 25 May 2026 05:56:18 +0000 (08:56 +0300)
commite2ef8fe42ccef597bfeab901dd6e39589613b71e
treeb2824c40266db211a365246e3fd652b9f7f47976
parent6d57c26ef8cb9a3583993d8d6527858985c82c7b
server: fix checkpoints creation (#22929)

* common : add common_chat_split_by_role

* cont : fix spans to reach end of message

* server: fix checkpoints creation

- extract message_spans from chat templates
- find the prompt token position before the latest user message
- split prompt batching at that position
- create a context checkpoint before the latest user input
- avoid periodic mid-prompt checkpoints when that position is known
- handle multimodal prompts when mapping text/template positions to server prompt tokens
- add --checkpoint-min-step to control minimum spacing between checkpoints

* cont : clean-up

* Support autoparser detection for message barriers

* server: fix message span delimiter and update docs

---------

Co-authored-by: Alde Rojas <redacted>
Co-authored-by: Georgi Gerganov <redacted>
Co-authored-by: Piotr Wilkin <redacted>
15 files changed:
common/arg.cpp
common/chat-auto-parser-helpers.cpp
common/chat-auto-parser.h
common/chat-diff-analyzer.cpp
common/chat.cpp
common/chat.h
common/common.cpp
common/common.h
tests/test-chat-auto-parser.cpp
tests/test-chat.cpp
tools/cli/README.md
tools/server/README.md
tools/server/server-common.cpp
tools/server/server-context.cpp
tools/server/server-task.h