]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
Fix gemma2 tokenizer convert (#8244)
authorXuan Son Nguyen <redacted>
Mon, 1 Jul 2024 23:07:23 +0000 (01:07 +0200)
committerGitHub <redacted>
Mon, 1 Jul 2024 23:07:23 +0000 (01:07 +0200)
commit5fac350b9cc49d0446fc291b9c4ad53666c77591
treeadb586ab484707012331235920c9ade41abc8f33
parentcb5fad4c6c2cbef92e9b8b63449e1cb7664e4846
Fix gemma2 tokenizer convert (#8244)

* fix gemma2 tokenizer convert

* remove scores

* improve code, fix new line issue
convert-hf-to-gguf.py