From: ynankani <redacted>
Date: Thu, 23 Apr 2026 05:19:51 +0000 (+0000)
Subject: convert : Handle ModelOpt produced mixed precision model during convert to GGUF ... 
X-Git-Tag: upstream/0.0.10438~1541
X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=5eaee65384f8bcf3564a92c7f9c33b5d80b6267f;p=pkg%2Fggml%2Fsources%2Fllama.cpp

convert : Handle ModelOpt produced mixed precision model during convert to GGUF (#22247)

* Handle ModelOpt produced mixed precision model during convert to GGUF

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <redacted>

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <redacted>

---------

Co-authored-by: Sigbjørn Skjæret <redacted>
---

diff --git a/convert_hf_to_gguf.py b/convert_hf_to_gguf.py
index 090686b15..93d5509e6 100755
--- a/convert_hf_to_gguf.py
+++ b/convert_hf_to_gguf.py
@@ -746,7 +746,12 @@ class ModelBase:
 
         if (not quant_algo or not quant_layers) and quant_config_file.is_file():
             with open(quant_config_file, "r", encoding="utf-8") as f:
-                quant_config = json.load(f).get("quantization") or {}
+                hf_quant_config = json.load(f)
+                quant_config = hf_quant_config.get("quantization") or {}
+                producer = hf_quant_config.get("producer") or {}
+                producer_name = (producer.get("name") or "").lower()
+                if quant_method is None:
+                    self.hparams.setdefault("quantization_config", {})["quant_method"] = producer_name
                 quant_algo = quant_config.get("quant_algo", quant_algo)
                 quant_layers = quant_config.get("quantized_layers", quant_layers) or {}