]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
overview / pkg / ggml / sources / llama.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 5cb63e2)
sentencepiece bpe compatible tokenizer (#252)
authorMack Straight <redacted>
Mon, 20 Mar 2023 10:17:23 +0000 (03:17 -0700)
committerGitHub <redacted>
Mon, 20 Mar 2023 10:17:23 +0000 (03:17 -0700)
commit074bea2eb1f1349a0118239c4152914aecaa1be4
tree41ce911ac28d858cabfeff650b10521b30838656tree
parent5cb63e2493c49bc2c3b9b355696e8dc26cdd0380commit | diff
sentencepiece bpe compatible tokenizer (#252)

* potential out of bounds read

* fix quantize

* style

* Update convert-pth-to-ggml.py

* mild cleanup

* don't need the space-prefixing here rn since main.cpp already does it

* new file magic + version header field

* readme notice

* missing newlines

Co-authored-by: slaren <redacted>
Makefile diff | blob | history
README.md diff | blob | history
convert-pth-to-ggml.py diff | blob | history
main.cpp diff | blob | history
quantize.cpp diff | blob | history
utils.cpp diff | blob | history
utils.h diff | blob | history
Packaging of ggml-org/llama.cpp