]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
Fix and restrict NVFP4 edge-cases in llama-graph (#24331)
authorOliver Simons <redacted>
Tue, 16 Jun 2026 09:52:38 +0000 (11:52 +0200)
committerGitHub <redacted>
Tue, 16 Jun 2026 09:52:38 +0000 (11:52 +0200)
commit02810c7aa89b8100b90b7b0f5e96bc55aafd3d0a
tree96c20e307b352f170bf006ab8bf12f4d792ce784
parenta1824902b573134458945b0c7973e105a7837b59
Fix and restrict NVFP4 edge-cases in llama-graph (#24331)

* Move post-GEMM MUL required for dequant b4 lora and bias add

see https://github.com/ggml-org/llama.cpp/pull/23484 :
1. For lora, I would presume we want fully dequantized values before
   doing the residuals, but this depends on how the LORAs were
generated. Literature tells me LORA happens post-mul but pre-bias add https://github.com/ggml-org/llama.cpp/pull/8332
2. For ModelOPT, bias-add should happen on [fully-dequantized
   values](https://github.com/NVIDIA/Model-Optimizer/blob/b49f9b9e2d747af992d78a3aa7f10efe5a8847e1/modelopt/torch/quantization/backends/nvfp4_gemm.py#L59-L64)

* Restrict build_ffn for NVFP4 to supported combinations
src/llama-graph.cpp
src/llama-graph.h