From: Yash Raj Pandey <redacted>
Date: Sat, 18 Jul 2026 11:43:18 +0000 (-0400)
Subject: llama-quant : exclude i32 ffn_gate_tid2eid routing table from quantization (#25787)
X-Git-Tag: upstream/0.0.10438~371
X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=4937ca83f4f3da63004943fe05d8aa4f0217d238;p=pkg%2Fggml%2Fsources%2Fllama.cpp

llama-quant : exclude i32 ffn_gate_tid2eid routing table from quantization (#25787)

DeepSeek-V4's ffn_gate_tid2eid tensor is an i32 token-id -> expert-id
index table, not weights. It was never added to the name-based
exclusion list alongside ffn_gate_inp.weight, so llama-quantize tries
to quantize it and fails since i32 cannot convert to a float type.

Fixes ggml-org/llama.cpp#25754

Signed-off-by: Yash Raj Pandey <redacted>
---

diff --git a/src/llama-quant.cpp b/src/llama-quant.cpp
index b66759b27..b187f5de9 100644
--- a/src/llama-quant.cpp
+++ b/src/llama-quant.cpp
@@ -306,6 +306,9 @@ static bool tensor_allows_quantization(const llama_model_quantize_params * param
     // NOTE: can't use LLM_TN here because the layer number is not known
     quantize &= name.find("ffn_gate_inp.weight") == std::string::npos;
 
+    // do not quantize the i32 token-id -> expert-id routing table (DeepSeek-V4)
+    quantize &= name.find("ffn_gate_tid2eid.weight") == std::string::npos;
+
     // these are very small (e.g. 4x4)
     quantize &= name.find("altup")  == std::string::npos;
     quantize &= name.find("laurel") == std::string::npos;