]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
overview / pkg / ggml / sources / llama.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 1bbdaf6)
grammars: 1.5x faster inference w/ complex grammars (vector reserves / reuses) (...
authorOlivier Chafik <redacted>
Thu, 11 Apr 2024 18:47:34 +0000 (19:47 +0100)
committerGitHub <redacted>
Thu, 11 Apr 2024 18:47:34 +0000 (19:47 +0100)
commitcbaadc92942c50aab599a9e4c163afc1f44f7c26
tree0a4b962430740a81a6b1789f1edd9ee50074dde3tree
parent1bbdaf6ecda6f0a360dfb307b256fcb6838c560bcommit | diff
grammars: 1.5x faster inference w/ complex grammars (vector reserves / reuses) (#6609)

* grammars: reserve rejects & next candidates

* grammars: reuse new_stacks

* grammars: fix missing sig change in llama.h

* grammars: fix test (api changed)

* grammars: update gbnf-validator.cpp

* grammars: simpler syntax (no swap)
examples/gbnf-validator/gbnf-validator.cpp diff | blob | history
llama.cpp diff | blob | history
llama.h diff | blob | history
tests/test-grammar-integration.cpp diff | blob | history
Packaging of ggml-org/llama.cpp