]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
overview / pkg / ggml / sources / llama.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 044ec4b)
llama : support models without vocabulary (#5798)
authorMichael Podvitskiy <redacted>
Thu, 14 Mar 2024 16:21:56 +0000 (17:21 +0100)
committerGitHub <redacted>
Thu, 14 Mar 2024 16:21:56 +0000 (18:21 +0200)
commit69ff61397d2b7b550dcdda4a35b35128892408b0
tree70b5bac95a58dcf734e8035c5fd47b92ceb510a1tree
parent044ec4b2a567f649459ccd20af2f387c784faa51commit | diff
llama : support models without vocabulary (#5798)

* additional methods to read model and ctx parameters

* vocab size as a part of a model metadata

* models without vocabulary, convert.py part

* models without vocabulary, llama.cpp part

* PR clean up

* converter scrypt fixes

* llama_vocab_type update (renamed the new key)

* pr review fixes

* revert function renaming

* one more NoVocab assert
convert.py diff | blob | history
gguf-py/gguf/constants.py diff | blob | history
gguf-py/gguf/gguf_writer.py diff | blob | history
llama.cpp diff | blob | history
llama.h diff | blob | history
Packaging of ggml-org/llama.cpp