From: slaren <redacted>
Date: Sun, 19 Nov 2023 08:37:08 +0000 (+0100)
Subject: gguf : add tokenizer.chat_template documentation (#616)
X-Git-Tag: upstream/0.0.1642~1191
X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=57c468b8655f3630d174950e46ae2eb1fd5042f4;p=pkg%2Fggml%2Fsources%2Fggml

gguf : add tokenizer.chat_template documentation (#616)
---

diff --git a/docs/gguf.md b/docs/gguf.md
index 75610a45..794a5292 100644
--- a/docs/gguf.md
+++ b/docs/gguf.md
@@ -512,6 +512,7 @@ Hugging Face maintains their own `tokenizers` library that supports a wide varie
 Other tokenizers may be used, but are not necessarily standardized. They may be executor-specific. They will be documented here as they are discovered/further developed.
 
 - `tokenizer.rwkv.world: string`: a RWKV World tokenizer, like [this](https://github.com/BlinkDL/ChatRWKV/blob/main/tokenizer/rwkv_vocab_v20230424.txt). This text file should be included verbatim.
+- `tokenizer.chat_template : string`: a Jinja template that specifies the input format expected by the model. For more details see: <https://huggingface.co/docs/transformers/main/en/chat_templating>
 
 ### Computation graph