]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
convert : fix gemma v1 tokenizer convert (#8248)
authorGeorgi Gerganov <redacted>
Thu, 4 Jul 2024 07:41:03 +0000 (10:41 +0300)
committerGitHub <redacted>
Thu, 4 Jul 2024 07:41:03 +0000 (10:41 +0300)
commit20fc3804bfb727074bc270b6eacb60af8d0bf7d4
tree068ffda2c36a3f39dfe59d43399f7fd31648f859
parentf619024764e72261f14d7c31d892b8fb976603b4
convert : fix gemma v1 tokenizer convert (#8248)

ggml-ci
28 files changed:
convert-hf-to-gguf-update.py
convert-hf-to-gguf.py
models/ggml-vocab-bert-bge.gguf.inp
models/ggml-vocab-bert-bge.gguf.out
models/ggml-vocab-command-r.gguf.inp
models/ggml-vocab-command-r.gguf.out
models/ggml-vocab-deepseek-coder.gguf.inp
models/ggml-vocab-deepseek-coder.gguf.out
models/ggml-vocab-deepseek-llm.gguf.inp
models/ggml-vocab-deepseek-llm.gguf.out
models/ggml-vocab-falcon.gguf.inp
models/ggml-vocab-falcon.gguf.out
models/ggml-vocab-gpt-2.gguf.inp
models/ggml-vocab-gpt-2.gguf.out
models/ggml-vocab-llama-bpe.gguf.inp
models/ggml-vocab-llama-bpe.gguf.out
models/ggml-vocab-llama-spm.gguf.inp
models/ggml-vocab-llama-spm.gguf.out
models/ggml-vocab-mpt.gguf.inp
models/ggml-vocab-mpt.gguf.out
models/ggml-vocab-phi-3.gguf.inp
models/ggml-vocab-phi-3.gguf.out
models/ggml-vocab-qwen2.gguf.inp
models/ggml-vocab-qwen2.gguf.out
models/ggml-vocab-refact.gguf.inp
models/ggml-vocab-refact.gguf.out
models/ggml-vocab-starcoder.gguf.inp
models/ggml-vocab-starcoder.gguf.out