From: woodx <redacted>
Date: Fri, 7 Jun 2024 07:09:45 +0000 (+0800)
Subject: server : do not get prompt in infill mode (#7286)
X-Git-Tag: upstream/0.0.4488~1383
X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=a5cabd76491f07494c5b8267f921c73f5e2bbfb4;p=pkg%2Fggml%2Fsources%2Fllama.cpp

server : do not get prompt in infill mode (#7286)

* avoid to get prompt in infill mode and embedding mode

* remove embedding mode

* refactor format

---------

Co-authored-by: wudexiang <redacted>
---

diff --git a/examples/server/server.cpp b/examples/server/server.cpp
index 74da81da..52822060 100644
--- a/examples/server/server.cpp
+++ b/examples/server/server.cpp
@@ -888,7 +888,7 @@ struct server_context {
         slot.params.input_suffix = json_value(data, "input_suffix", default_params.input_suffix);
 
         // get prompt
-        {
+        if (!task.infill) {
             const auto & prompt = data.find("prompt");
             if (prompt == data.end()) {
                 send_error(task, "Either \"prompt\" or \"messages\" must be provided", ERROR_TYPE_INVALID_REQUEST);