git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit

]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit

overview / pkg / ggml / sources / llama.cpp / commit

author	Kerfuffle <redacted>
	Fri, 17 Nov 2023 02:14:37 +0000 (19:14 -0700)
committer	GitHub <redacted>
	Fri, 17 Nov 2023 02:14:37 +0000 (19:14 -0700)
commit	91f6499393d2d999331fbfdba47a7f8b9f913f0d
tree	27caf3ad0b9cec979bb5ed3317b5334bdcd9470c	tree
parent	8da46278e1a57107591653275f8e03a281de94f0	commit \| diff

Respect tokenizer.ggml.add_bos_token value when tokenizing (#4040)

* gguf-py: gguf-dump: Respect --no-tensor flag in JSON mode.

* Respect add_bos_token GGUF metadata value

* gguf-py: Try to fix SpecialVocab giving up too easily for the Nth time

12 files changed:

Packaging of ggml-org/llama.cpp

RSS Atom

common/common.cpp		diff \| blob \| history
common/common.h		diff \| blob \| history
examples/infill/infill.cpp		diff \| blob \| history
examples/llava/llava-cli.cpp		diff \| blob \| history
examples/main/main.cpp		diff \| blob \| history
examples/perplexity/perplexity.cpp		diff \| blob \| history
examples/server/server.cpp		diff \| blob \| history
gguf-py/gguf/vocab.py		diff \| blob \| history
gguf-py/pyproject.toml		diff \| blob \| history
gguf-py/scripts/gguf-dump.py		diff \| blob \| history
llama.cpp		diff \| blob \| history
llama.h		diff \| blob \| history