From: Kerfuffle <redacted>
Date: Thu, 24 Aug 2023 16:11:13 +0000 (-0600)
Subject: Fix for main example getting stuck when -n -2 and --interactive (#2767)
X-Git-Tag: gguf-v0.4.0~256
X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=7694adda8d1111b3cf758ad6c91d754a0a4cacff;p=pkg%2Fggml%2Fsources%2Fllama.cpp

Fix for main example getting stuck when -n -2 and --interactive (#2767)

* Fix for main example getting stuck when -n -2 and --interactive

* Add a comment so future generations may suffer less.
---

diff --git a/examples/main/main.cpp b/examples/main/main.cpp
index 1393f0b0..cb8747c2 100644
--- a/examples/main/main.cpp
+++ b/examples/main/main.cpp
@@ -798,7 +798,8 @@ int main(int argc, char ** argv) {
         }
 
         // In interactive mode, respect the maximum number of tokens and drop back to user input when reached.
-        if (params.interactive && n_remain <= 0 && params.n_predict != -1) {
+        // We skip this logic when n_predict == -1 (infinite) or -2 (stop at context size).
+        if (params.interactive && n_remain <= 0 && params.n_predict >= 0) {
             n_remain = params.n_predict;
             is_interacting = true;
         }