]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
ggml-cuda: Add generic NVFP4 MMQ kernel (llama/21074)
authorMichael Wand <redacted>
Wed, 1 Apr 2026 10:04:58 +0000 (03:04 -0700)
committerGeorgi Gerganov <redacted>
Thu, 30 Apr 2026 08:28:59 +0000 (11:28 +0300)
commit981195be5aa6ec359b2e536b4194c0f7a7a3ee20
tree0debcc3965791bb8db506862036b83bad3edbaf4
parentace95aac6b32f6e0e57a45789d3ec82c8c89e9ac
ggml-cuda: Add generic NVFP4 MMQ kernel (llama/21074)

* Introduced NVFP4 generic MMQ kernel

* Added extra FP8 guard, hope to solve ci HIP failure

* Rename tiles and use HIP_FP8_AVAILABLE

* Removed remaning FP8 straggler and added const int

* Const

* Removed DECL_MMQ_CASE artifact

* Removed newline

* Removed space after else

* Changed HIP FP8 NVFP4 conversion gate

* Added new line to bottom of mmq.cu 270

* Removed extra spaces

* Removed single space in front of else on line 814

* Added NVFP4 to generate cu script so HIP can see it, further tightened logic

* Include generated mmq-instance-nvfp4.cu

* Added NVFP4 mmq to HIP Check ignore list

* Update ggml/src/ggml-cuda/mmq.cuh

Changed to Q3_K tile to read MMQ_MMA_TILE_X_K_NVFP4

Co-authored-by: Johannes Gäßler <redacted>
* Update ggml/src/ggml-cuda/mmq.cuh

Changed to Q3_K tile to read MMQ_MMA_TILE_X_K_NVFP4 in tile assert

Co-authored-by: Johannes Gäßler <redacted>
* Update ggml/src/ggml-cuda/mmq.cuh

Added function name ending for end if

Co-authored-by: Johannes Gäßler <redacted>
* Added function names to closing endif

Co-authored-by: Johannes Gäßler <redacted>
---------

Co-authored-by: Johannes Gäßler <redacted>
ggml/src/ggml-cuda/common.cuh
ggml/src/ggml-cuda/ggml-cuda.cu
ggml/src/ggml-cuda/mmq.cu
ggml/src/ggml-cuda/mmq.cuh
ggml/src/ggml-cuda/template-instances/generate_cu_files.py
ggml/src/ggml-cuda/template-instances/mmq-instance-nvfp4.cu [new file with mode: 0644]