]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
ggml-cuda: ds_read_b128 for q4_0 and q4_1 mmq kernels (#21168)
authoriacopPBK <redacted>
Tue, 7 Apr 2026 19:47:42 +0000 (21:47 +0200)
committerGitHub <redacted>
Tue, 7 Apr 2026 19:47:42 +0000 (21:47 +0200)
commit66c4f9ded01b29d9120255be1ed8d5835bcbb51d
treea74193d6bdea21b5642656c0a139460cfa4e2679
parent93bdc6156333082421d82b52d36f7dacb0542495
ggml-cuda: ds_read_b128 for q4_0 and q4_1 mmq kernels (#21168)

* ds_read_b128 for q4_0 and q4_1 mmq kernels

     Current for loop generates ds_read_b32 instructions with hip compiler, the new solution generates ds_read_b128 instructions for the same operation, saving some LDS bandwidth. Tested on MI50 and RX6800XT, its faster on both.

* Vectorized lds load update: used ggml_cuda_get_max_cpy_bytes and ggml_cuda_memcpy_1 functions for generic implementation

* Explicit for loop in mmq, renamed vec into tmp

* Fixed max_cpy usage in the loading loop

* Fixed typo in q4_1 kernel

* Update ggml/src/ggml-cuda/mmq.cuh

Co-authored-by: Johannes Gäßler <redacted>
* Update ggml/src/ggml-cuda/mmq.cuh

Co-authored-by: Johannes Gäßler <redacted>
* Update ggml/src/ggml-cuda/mmq.cuh

Co-authored-by: Johannes Gäßler <redacted>
* Renoved trailing white line 500

* Update mmq.cuh removed other whitelines

* Remove trailing whitespaces

---------

Co-authored-by: iacopPBK <redacted>
Co-authored-by: Johannes Gäßler <redacted>
Co-authored-by: iacopPBK <redacted>
ggml/src/ggml-cuda/mmq.cuh