From: Dr. Tom Murphy VII Ph.D <redacted>
Date: Sat, 26 Aug 2023 18:12:56 +0000 (-0400)
Subject: main : fix bug (penalize_nl=false doesn't work) + suppress warning on mingw (#1528)
X-Git-Tag: gguf-v0.4.0~228
X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=72f895c923ba98b8f2af294440206f35915c0501;p=pkg%2Fggml%2Fsources%2Fllama.cpp

main : fix bug (penalize_nl=false doesn't work) + suppress warning on mingw (#1528)

* Fix bug in main.cpp where penalize_nl=false has no effect. It modifies the underlying logits array, but at this point we are already working on the candidates copy.

* Suppress redefinition warning for NOMINMAX on mingw. In my installation, this macro is already defined by /usr/lib/gcc/x86_64-w64-mingw32/11/include/c++/x86_64-w64-mingw32/bits/os_defines.h:45.

* main : fix indentation

* main : pass ctx to llama_token_nl()

---------

Co-authored-by: Georgi Gerganov <redacted>
---

diff --git a/examples/main/main.cpp b/examples/main/main.cpp
index 4665b82f..11d7a7e4 100644
--- a/examples/main/main.cpp
+++ b/examples/main/main.cpp
@@ -604,7 +604,12 @@ int main(int argc, char ** argv) {
                     last_n_tokens.data() + last_n_tokens.size() - last_n_repeat,
                     last_n_repeat, alpha_frequency, alpha_presence);
                 if (!penalize_nl) {
-                    logits[llama_token_nl(ctx)] = nl_logit;
+                    for (size_t idx = 0; idx < candidates_p.size; idx++) {
+                        if (candidates_p.data[idx].id == llama_token_nl(ctx)) {
+                            candidates_p.data[idx].logit = nl_logit;
+                            break;
+                        }
+                    }
                 }
 
                 if (grammar != NULL) {