From: Ivan Stepanov <redacted>
Date: Wed, 5 Apr 2023 16:20:05 +0000 (+0300)
Subject: llama : define non-positive top_k; top_k range check (#779)
X-Git-Tag: gguf-v0.4.0~1011
X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=5a8c4f624077373a198cd562146ffa67b02ebc75;p=pkg%2Fggml%2Fsources%2Fllama.cpp

llama : define non-positive top_k; top_k range check (#779)

* Define non-positive top_k; top_k range check

* minor : brackets

---------

Co-authored-by: Georgi Gerganov <redacted>
---

diff --git a/llama.cpp b/llama.cpp
index 854bb899..e4517959 100644
--- a/llama.cpp
+++ b/llama.cpp
@@ -1229,7 +1229,9 @@ static llama_vocab::id llama_sample_top_p_top_k(
         }
     }
 
-    sample_top_k(logits_id, top_k);
+    if (top_k > 0 && top_k < n_logits) {
+        sample_top_k(logits_id, top_k);
+    }
 
     float maxl = -std::numeric_limits<float>::infinity();
     for (const auto & kv : logits_id) {