]>
git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
llama : support Llama 3 HF conversion (#6745)
* Support Llama 3 conversion
The tokenizer is BPE.
* style
* Accept suggestion
Co-authored-by: Sourab Mangrulkar <redacted>
* llama : add llama_token_is_eog()
ggml-ci
* llama : auto-detect more EOT tokens when missing in KV data
* convert : replacing EOS token is a hack
* llama : fix codegemma EOT token + add TODOs
* llama : fix model type string for 8B model
---------
Co-authored-by: Sourab Mangrulkar <redacted>
Co-authored-by: Georgi Gerganov <redacted>
20 files changed:
overview / pkg / ggml / sources / llama.cpp / commit