]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
overview / pkg / ggml / sources / llama.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: dc68f00)
Add byte token type when tokenizer.model is not exists (#4641)
authorwonjun Jang <redacted>
Wed, 27 Dec 2023 08:37:25 +0000 (17:37 +0900)
committerGitHub <redacted>
Wed, 27 Dec 2023 08:37:25 +0000 (17:37 +0900)
commitf56d6077d0c37e6606ac0a4fa3169de70593acfe
tree1c7eca9489b2bb8e2723f3d005c18770d58fa9a8tree
parentdc68f0054cd279cddddb0cae0c9ef4f9cbaa512acommit | diff
Add byte token type when tokenizer.model is not exists (#4641)

* Add byte token type to hf format

* remove unused variable
convert.py diff | blob | history
Packaging of ggml-org/llama.cpp