]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
Write an optimized flash_attn_stream_k_fixup kernel (llama/21159)
authorGaurav Garg <redacted>
Mon, 6 Apr 2026 18:34:29 +0000 (00:04 +0530)
committerGeorgi Gerganov <redacted>
Thu, 30 Apr 2026 08:29:01 +0000 (11:29 +0300)
commit7b19b94c5dc822a21bdb2e574ece4a7b2316c436
treee426f7cf6af61d65bad625c25d644cd0be8d42af
parent42e4a28865c6909d8a5b6390a68740404005aa3f
Write an optimized flash_attn_stream_k_fixup kernel (llama/21159)

* Write an optimized flash_attn_stream_k_fixup kernel

Write a specialized and more optimized kernel for cases where nblocks_stream_k is multiple of ntiles_dst.
Make nblocks_stream_k to multiple of ntiles_dst if nblocks_stream_k > 2 * ntiles_dst

* Use the new kernel only for nblocks_stream_k_raw > 4 * ntiles_dst to make sure we have enough concurrency on GPUs

* Address review comments

* Address review comments

* Revert variable names to original
ggml/src/ggml-cuda/fattn-common.cuh