From: Han Qingzhe <redacted>
Date: Wed, 26 Nov 2025 20:44:07 +0000 (+0800)
Subject: clip: (minicpmv) fix resampler kq_scale (#17516)
X-Git-Tag: upstream/0.0.7446~279
X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=1d594c295c8863e9077feeb50305a5e19493e6ee;p=pkg%2Fggml%2Fsources%2Fllama.cpp

clip: (minicpmv) fix resampler kq_scale (#17516)

* debug:"solve minicpmv precision problem"

* “debug minicpmv”

* Apply suggestion from @ngxson

---------

Co-authored-by: Xuan-Son Nguyen <redacted>
---

diff --git a/tools/mtmd/clip.cpp b/tools/mtmd/clip.cpp
index abdb778f..52ea542d 100644
--- a/tools/mtmd/clip.cpp
+++ b/tools/mtmd/clip.cpp
@@ -1175,10 +1175,11 @@ struct clip_graph {
             cb(K, "resampler_K", -1);
             cb(V, "resampler_V", -1);
 
+            float resampler_kq_scale = 1.0f/ sqrtf(float(d_head));
             embeddings = build_attn(
                 model.mm_model_attn_o_w,
                 model.mm_model_attn_o_b,
-                Q, K, V, nullptr, kq_scale, -1);
+                Q, K, V, nullptr, resampler_kq_scale, -1);
             cb(embeddings, "resampler_attn_out", -1);
         }
         // layernorm