From: klosax <redacted>
Date: Wed, 23 Aug 2023 14:46:03 +0000 (+0200)
Subject: main : insert bos if no tokens (#2727)
X-Git-Tag: gguf-v0.4.0~269
X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=5290c38e6e9b66ee2b543e560e301c1a1a90929c;p=pkg%2Fggml%2Fsources%2Fllama.cpp

main : insert bos if no tokens (#2727)

* main.cpp : insert bos if no tokens

* Update examples/main/main.cpp

* Update examples/main/main.cpp

---------

Co-authored-by: Georgi Gerganov <redacted>
---

diff --git a/examples/main/main.cpp b/examples/main/main.cpp
index 388e1f7d..0a22f3c2 100644
--- a/examples/main/main.cpp
+++ b/examples/main/main.cpp
@@ -197,6 +197,11 @@ int main(int argc, char ** argv) {
         embd_inp = session_tokens;
     }
 
+    // Should not run without any tokens
+    if (embd_inp.empty()) {
+        embd_inp.push_back(llama_token_bos(ctx));
+    }
+
     // Tokenize negative prompt
     std::vector<llama_token> guidance_inp;
     int guidance_offset = 0;