]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
overview / pkg / ggml / sources / whisper.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 77b635e)
ggml-cuda: native bf16 flash attention for vec kernel (llama/20525)
authorPatrick Buckley <redacted>
Sun, 22 Mar 2026 10:05:51 +0000 (03:05 -0700)
committerGeorgi Gerganov <redacted>
Sun, 29 Mar 2026 12:04:36 +0000 (15:04 +0300)
commit69f0d907ee609091eaa3a552ccfd63c9390e55d6
treea1843b06fe5d79fd65b4f860d452675aecb075f3tree
parent77b635e9c4f0e3a8fe0252f2197f61b37a62a22ccommit | diff
ggml-cuda: native bf16 flash attention for vec kernel (llama/20525)

* ggml-cuda: native bf16 flash attention for vec and tile kernels

mma kernel still converts bf16 to fp16 before launch, native mma bf16 todo

* ggml-cuda: address code owner review feedback

reverted tile kernel changes to avoid larger refactor

* fix ci failures on turing and hip

* fix bf16 vec kernel compile on hip v_dot2 platforms

* add comments

---------

Co-authored-by: Johannes Gäßler <redacted>
21 files changed:
ggml/src/ggml-cuda/CMakeLists.txt diff | blob | history
ggml/src/ggml-cuda/convert.cuh diff | blob | history
ggml/src/ggml-cuda/fattn-common.cuh diff | blob | history
ggml/src/ggml-cuda/fattn-vec.cuh diff | blob | history
ggml/src/ggml-cuda/fattn.cu diff | blob | history
ggml/src/ggml-cuda/template-instances/fattn-vec-instance-bf16-bf16.cu [new file with mode: 0644] blob
ggml/src/ggml-cuda/template-instances/fattn-vec-instance-bf16-f16.cu [new file with mode: 0644] blob
ggml/src/ggml-cuda/template-instances/fattn-vec-instance-bf16-q4_0.cu [new file with mode: 0644] blob
ggml/src/ggml-cuda/template-instances/fattn-vec-instance-bf16-q4_1.cu [new file with mode: 0644] blob
ggml/src/ggml-cuda/template-instances/fattn-vec-instance-bf16-q5_0.cu [new file with mode: 0644] blob
ggml/src/ggml-cuda/template-instances/fattn-vec-instance-bf16-q5_1.cu [new file with mode: 0644] blob
ggml/src/ggml-cuda/template-instances/fattn-vec-instance-bf16-q8_0.cu [new file with mode: 0644] blob
ggml/src/ggml-cuda/template-instances/fattn-vec-instance-f16-bf16.cu [new file with mode: 0644] blob
ggml/src/ggml-cuda/template-instances/fattn-vec-instance-q4_0-bf16.cu [new file with mode: 0644] blob
ggml/src/ggml-cuda/template-instances/fattn-vec-instance-q4_1-bf16.cu [new file with mode: 0644] blob
ggml/src/ggml-cuda/template-instances/fattn-vec-instance-q5_0-bf16.cu [new file with mode: 0644] blob
ggml/src/ggml-cuda/template-instances/fattn-vec-instance-q5_1-bf16.cu [new file with mode: 0644] blob
ggml/src/ggml-cuda/template-instances/fattn-vec-instance-q8_0-bf16.cu [new file with mode: 0644] blob
ggml/src/ggml-cuda/template-instances/generate_cu_files.py diff | blob | history
ggml/src/ggml-hip/CMakeLists.txt diff | blob | history
ggml/src/ggml-musa/CMakeLists.txt diff | blob | history
Packaging of ggerganov/whisper.cpp