]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
grammars: x{min,max} repetition operator (#6640)
authorOlivier Chafik <redacted>
Thu, 6 Jun 2024 09:07:06 +0000 (10:07 +0100)
committerGitHub <redacted>
Thu, 6 Jun 2024 09:07:06 +0000 (10:07 +0100)
commit55b2d0849d3ec9e45e4a4d9e480f5fa7977872a6
treeaef3a9e9a051670751912daf0d06a76783b44b45
parentf5d7b268ec4bf8628aa6ccc9f6631d0230dde76f
grammars: x{min,max} repetition operator (#6640)

* grammars: x{min,max} repetition operator + tweak +/*/? to avoid duplication of original over alternates

* grammars: handle `x{n}` and fix `x{n,n}`

* grammars: document new repetition operators

* grammars: uniform use of int for min & max

* grammars: refactor parser test

* grammar: parsing tests w/ natural pretty print of updated expectations

* grammars: much prettier print of expectations (+ TEST_GRAMMAR_PARSER_PRINT_ALL=1 to force all)

* grammars: improve test pretty print again

* grammars: pretty print rules and chars

* grammars: fix copy rule skipping

* grammars: disallow `a{,}` (not allowed in regexps)

* Update common/grammar-parser.cpp

Co-authored-by: Clint Herron <redacted>
* grammars: fix copy rule skipping (again) & display of expectations

* grammars: more test cases

* grammars: update reps parsing to bring ? / * / + closer to before

* json: use new GBNF repetitions{m,n} syntax

* grammars: update performance gotchas w/ repetition advice

* Update examples/json_schema_to_grammar.py

Co-authored-by: Clint Herron <redacted>
* Update examples/server/public/json-schema-to-grammar.mjs

Co-authored-by: Clint Herron <redacted>
* grammars: comment on rule repetitions

* grammars: ensure unambiguous number alternatives

* grammar: nit typo switched error msgs

* grammar: nit numbering in comment

* json: update numeric rule to be unambiguous

* Apply suggestions from code review

Co-authored-by: Clint Herron <redacted>
* Update examples/server/public/json-schema-to-grammar.mjs

Co-authored-by: Clint Herron <redacted>
* json: fix integral-part

* grammar: add repetition tests

---------

Co-authored-by: Clint Herron <redacted>
common/grammar-parser.cpp
common/json-schema-to-grammar.cpp
examples/json_schema_to_grammar.py
examples/pydantic_models_to_grammar.py
examples/server/public/json-schema-to-grammar.mjs
grammars/README.md
tests/test-grammar-integration.cpp
tests/test-grammar-parser.cpp
tests/test-json-schema-to-grammar.cpp