From: Abhishek Gopinath K <redacted>
Date: Wed, 3 Apr 2024 15:42:52 +0000 (+0530)
Subject: Missing tokenizer.model error during gguf conversion (#6443)
X-Git-Tag: upstream/0.0.4488~1897
X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=db214fa578e00b01e0884fc2725c9349608bdab5;p=pkg%2Fggml%2Fsources%2Fllama.cpp

Missing tokenizer.model error during gguf conversion (#6443)

Co-authored-by: Jared Van Bortel <redacted>
---

diff --git a/convert-hf-to-gguf.py b/convert-hf-to-gguf.py
index afa034a8..bca1c2d7 100755
--- a/convert-hf-to-gguf.py
+++ b/convert-hf-to-gguf.py
@@ -323,8 +323,7 @@ class Model(ABC):
         toktypes: list[int] = []
 
         if not tokenizer_path.is_file():
-            print(f'Error: Missing {tokenizer_path}', file=sys.stderr)
-            sys.exit(1)
+            raise FileNotFoundError(f"File not found: {tokenizer_path}")
 
         tokenizer = SentencePieceProcessor(str(tokenizer_path))
         vocab_size = self.hparams.get('vocab_size', tokenizer.vocab_size())