]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
CUDA: Improve NVFP4 W4A4 activation quantization (#25730)
authorOliver Simons <redacted>
Wed, 22 Jul 2026 17:28:02 +0000 (19:28 +0200)
committerGitHub <redacted>
Wed, 22 Jul 2026 17:28:02 +0000 (19:28 +0200)
commit1a064ab0921238c1daa397d6f4a900ef33884de2
tree1e169065216612b6236a562f8840d5872736a511
parent0278d8362d78c5de291bc03b76016f7f74b2ab77
CUDA: Improve NVFP4 W4A4 activation quantization (#25730)

* Squash history before conflict-resolution during rebase on master

WIP commit

Add 32-byte loads, restore per-block amax

Use nvfp4x4 intrinsic when available

Fuse per-channel amax and quantization kernels

Do pointer arithmetic only once on x

Remove unnecessary ternary in the load

We assert on host side that ne00 is 64-aligned

Add back scale-search, but optimize it with intrinsics

Code cleanup

Make scale in MMQ-epilogue NVFP4-specific/restrictive for now

Remove unneeded include, add comment

Fix trailing whitespace

Guard __builtin_align__(32) struct to NVIDIA

Seems like HIP doesn't have this available, see https://github.com/ggml-org/llama.cpp/actions/runs/29438651734/job/87431623001

* compiler massaging to avoid unnecessary LDCs

* kvalues_mxfp4 -> kvalues_nvfp4 in quantize_mmq_nvfp4

* Always pass in src1_scale.ptr

* Extract ggml_cuda_is_aligned helper
ggml/src/ggml-cuda/common.cuh
ggml/src/ggml-cuda/mmq.cu
ggml/src/ggml-cuda/mmq.cuh
ggml/src/ggml-cuda/quantize.cu
ggml/src/ggml-cuda/quantize.cuh