]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
overview / pkg / ggml / sources / llama.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 037259b)
llama : allow raw byte in SPM vocabs; don't crash on nl 404 (#5478)
authorAarni Koskela <redacted>
Tue, 13 Feb 2024 16:18:16 +0000 (18:18 +0200)
committerGitHub <redacted>
Tue, 13 Feb 2024 16:18:16 +0000 (18:18 +0200)
commitc4e6dd59e45ef7b14f7763fb073b517395dc176c
tree2cbd0de3c717bd1cc0fdf6f877035dcce6194d5btree
parent037259be689353081e7bae3c1ab4ab18e7fbe8c9commit | diff
llama : allow raw byte in SPM vocabs; don't crash on nl 404 (#5478)

* common : don't crash if newline token is not found

* common : llama_byte_to_token: allow falling back to finding just the token byte in SPM vocabs
llama.cpp diff | blob | history
Packaging of ggml-org/llama.cpp