]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
ggml-cuda: ds_read_b128 for q4_0 and q4_1 mmq kernels (llama/21168)
authoriacopPBK <redacted>
Tue, 7 Apr 2026 19:47:42 +0000 (21:47 +0200)
committerGeorgi Gerganov <redacted>
Thu, 30 Apr 2026 08:29:03 +0000 (11:29 +0300)
commitd91d1e8e6c362ac503cc659d5be8cafd7c35ab86
treedec1c012f7f32cdbf4ad318246195421617995ec
parentd1456437e1867fa957eb298648c68e48261ee476
ggml-cuda: ds_read_b128 for q4_0 and q4_1 mmq kernels (llama/21168)

* ds_read_b128 for q4_0 and q4_1 mmq kernels

     Current for loop generates ds_read_b32 instructions with hip compiler, the new solution generates ds_read_b128 instructions for the same operation, saving some LDS bandwidth. Tested on MI50 and RX6800XT, its faster on both.

* Vectorized lds load update: used ggml_cuda_get_max_cpy_bytes and ggml_cuda_memcpy_1 functions for generic implementation

* Explicit for loop in mmq, renamed vec into tmp

* Fixed max_cpy usage in the loading loop

* Fixed typo in q4_1 kernel

* Update ggml/src/ggml-cuda/mmq.cuh

Co-authored-by: Johannes Gäßler <redacted>
* Update ggml/src/ggml-cuda/mmq.cuh

Co-authored-by: Johannes Gäßler <redacted>
* Update ggml/src/ggml-cuda/mmq.cuh

Co-authored-by: Johannes Gäßler <redacted>
* Renoved trailing white line 500

* Update mmq.cuh removed other whitelines

* Remove trailing whitespaces

---------

Co-authored-by: iacopPBK <redacted>
Co-authored-by: Johannes Gäßler <redacted>
Co-authored-by: iacopPBK <redacted>
ggml/src/ggml-cuda/mmq.cuh