]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
vocab : refactor normalizer flags into options struct, add strip_accents (#24371)
authoro7si <redacted>
Thu, 11 Jun 2026 07:36:50 +0000 (15:36 +0800)
committerGitHub <redacted>
Thu, 11 Jun 2026 07:36:50 +0000 (10:36 +0300)
commit68f30663cf49102a85913ae6e278e6d1e9a1109e
tree01cfc6654ad0a2c7435286523239ce63eddfea7f
parentdb94854ff56549f62b84d2f31608259a9e5e0e9f
vocab : refactor normalizer flags into options struct, add strip_accents (#24371)

* vocab : refactor normalizer flags into options struct, add strip_accents

* Update src/llama-vocab.h

Co-authored-by: Sigbjørn Skjæret <redacted>
* Update src/llama-vocab.cpp

Co-authored-by: Sigbjørn Skjæret <redacted>
---------

Co-authored-by: Sigbjørn Skjæret <redacted>
gguf-py/gguf/constants.py
gguf-py/gguf/gguf_writer.py
gguf-py/gguf/vocab.py
src/llama-arch.cpp
src/llama-arch.h
src/llama-vocab.cpp
src/llama-vocab.h