]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
CUDA: experimental native mxfp4 support for blackwell (llama/17906)
authorAman Gupta <redacted>
Wed, 24 Dec 2025 14:28:26 +0000 (22:28 +0800)
committerGeorgi Gerganov <redacted>
Wed, 31 Dec 2025 10:39:43 +0000 (12:39 +0200)
commitf1f97081b7e530798015db17bb93080588c12c43
treeb9bb88707b4a4a5295e9cebd7ee99753be57513e
parent9e98a08ddbee18ce09933daad115cffa2239c861
CUDA: experimental native mxfp4 support for blackwell (llama/17906)

* CUDA: experimental native mxfp4 support for blackwell

* optimize load_tiles

* optimize quantize_mxfp4

* cleanup

* first pass review: formatting

* use interleaved layout for mma

* mmq: add assert for size

* use __nv_fp4x4_e2m1

* use iter_k as 512, cleanup

* Use 1200 as blackwell instead of 1000

* address review comments

* mmq: fix stride

* quantize.cu: use reference impl of e8m0 scale

* address review comments

* add 120f-virtual + minor fixes

---------

Co-authored-by: Aman Gupta <aman>
src/ggml-cuda/CMakeLists.txt
src/ggml-cuda/common.cuh
src/ggml-cuda/mma.cuh
src/ggml-cuda/mmq.cu
src/ggml-cuda/mmq.cuh
src/ggml-cuda/quantize.cu
src/ggml-cuda/quantize.cuh
src/ggml-cuda/vendors/cuda.h