]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
common : gracefully handle incomplete output (#20191)
authorAldehir Rojas <redacted>
Sun, 8 Mar 2026 16:17:02 +0000 (11:17 -0500)
committerGitHub <redacted>
Sun, 8 Mar 2026 16:17:02 +0000 (17:17 +0100)
commit451ef08432d1f7d3d6071d4006cbbeda21dcfbec
treedf7966f77579790bef0d71749cde7ae20a3a5d24
parent9b24886f78ce278b34186b47ed71a435f00d8d0d
common : gracefully handle incomplete output (#20191)

* common : handle incomplete UTF-8 at end of input in PEG parser

* cont : if reached end prematurely, emit needs_more_input to propagate partial output

* cont: refactor peg parse context to add lenient flag

* cont : remove partial flag, keep lenient flag
common/chat-peg-parser.cpp
common/chat-peg-parser.h
common/chat.cpp
common/peg-parser.cpp
common/peg-parser.h
tests/peg-parser/test-basic.cpp
tests/peg-parser/test-json-parser.cpp
tests/peg-parser/test-python-dict-parser.cpp
tests/peg-parser/test-unicode.cpp
tests/test-chat-auto-parser.cpp
tests/test-chat-peg-parser.cpp