]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
Inference support for T5 and FLAN-T5 model families (#5763)
authorfairydreaming <redacted>
Thu, 4 Jul 2024 13:46:11 +0000 (15:46 +0200)
committerGitHub <redacted>
Thu, 4 Jul 2024 13:46:11 +0000 (15:46 +0200)
commit807b0c49ff7071094f97ebc3a0a8e2b9e274f503
tree08af64d9adf5ef5f9413966c33bbcef1a009745e
parentf8c4c0738d72d2162736edd72dd5db8b269adca1
Inference support for T5 and FLAN-T5 model families (#5763)

* llama : add inference support and model types for T5 and FLAN-T5 model families

* llama : add new API functions to support encoder-decoder models: llama_encode(), llama_model_has_encoder(), llama_model_decoder_start_token()

* common, llama-cli, llama-batched : add support for encoder-decoder models

* convert-hf : handle shared token embeddings tensors in T5Model

* convert-hf : add support for SentencePiece BPE tokenizer in T5Model (for Pile-T5 models)

* convert-hf : add MT5ForConditionalGeneration and UMT5ForConditionalGeneration to architectures supported by T5Model

* convert : add t5 tokenizer tests, use "slow" HF tokenizer for t5

---------

Co-authored-by: Stanisław Szymczyk <redacted>
Co-authored-by: Georgi Gerganov <redacted>
33 files changed:
common/common.cpp
convert-hf-to-gguf-update.py
convert-hf-to-gguf.py
examples/batched/batched.cpp
examples/main/main.cpp
include/llama.h
models/ggml-vocab-bert-bge.gguf.inp
models/ggml-vocab-bert-bge.gguf.out
models/ggml-vocab-command-r.gguf.inp
models/ggml-vocab-command-r.gguf.out
models/ggml-vocab-deepseek-coder.gguf.inp
models/ggml-vocab-deepseek-coder.gguf.out
models/ggml-vocab-deepseek-llm.gguf.inp
models/ggml-vocab-deepseek-llm.gguf.out
models/ggml-vocab-falcon.gguf.inp
models/ggml-vocab-falcon.gguf.out
models/ggml-vocab-gpt-2.gguf.inp
models/ggml-vocab-gpt-2.gguf.out
models/ggml-vocab-llama-bpe.gguf.inp
models/ggml-vocab-llama-bpe.gguf.out
models/ggml-vocab-llama-spm.gguf.inp
models/ggml-vocab-llama-spm.gguf.out
models/ggml-vocab-mpt.gguf.inp
models/ggml-vocab-mpt.gguf.out
models/ggml-vocab-phi-3.gguf.inp
models/ggml-vocab-phi-3.gguf.out
models/ggml-vocab-qwen2.gguf.inp
models/ggml-vocab-qwen2.gguf.out
models/ggml-vocab-refact.gguf.inp
models/ggml-vocab-refact.gguf.out
models/ggml-vocab-starcoder.gguf.inp
models/ggml-vocab-starcoder.gguf.out
src/llama.cpp