From: Reguna <redacted>
Date: Thu, 18 Jun 2026 20:01:24 +0000 (+0800)
Subject: server: add "X-Accel-Buffering": "no" header to streaming endpoints (#24774)
X-Git-Tag: upstream/0.0.10438~724
X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=40f3aafc45990c1646397c67d2bd3e8eff3f543e;p=pkg%2Fggml%2Fsources%2Fllama.cpp

server: add "X-Accel-Buffering": "no" header to streaming endpoints (#24774)

* server: add "X-Accel-Buffering": "no" header to streaming endpoints

This header tells Nginx (as a reverse proxy) to NOT buffer responses. (only affects streaming endpoints)
Without it, Nginx will break streaming with certain applications (notably the Pi coding harness).
---

diff --git a/tools/server/server-http.cpp b/tools/server/server-http.cpp
index 5defee1f5..4f2abab00 100644
--- a/tools/server/server-http.cpp
+++ b/tools/server/server-http.cpp
@@ -492,6 +492,8 @@ using server_http_req_ptr = std::unique_ptr<server_http_req>;
 static void process_handler_response(server_http_req_ptr && request, server_http_res_ptr & response, httplib::Response & res) {
     if (response->is_stream()) {
         res.status = response->status;
+        // Tell Nginx to not buffer any streamed response
+        response->headers["X-Accel-Buffering"] = "no";
         set_headers(res, response->headers);
         const std::string content_type = response->content_type;
         // convert to shared_ptr as both chunked_content_provider() and on_complete() need to use it