]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
CUDA: faster q2_K, q3_K MMQ + int8 tensor cores (llama/7921)
authorJohannes Gäßler <redacted>
Fri, 14 Jun 2024 16:41:49 +0000 (18:41 +0200)
committerGeorgi Gerganov <redacted>
Sat, 15 Jun 2024 19:05:47 +0000 (22:05 +0300)
commita59a24f97da0d2c9a5aed04e75d4749d0f42f7ba
treeae8add773dc182cd5750464d369aa6e1eba797c3
parentee4f37c17d7d2310a7e2e1c06554f9d3ab6ef91a
CUDA: faster q2_K, q3_K MMQ + int8 tensor cores (llama/7921)

* CUDA: faster q2_K, q3_K MMQ + int8 tensor cores

* try CI fix

* try CI fix

* try CI fix

* fix data race

* rever q2_K precision related changes
src/ggml-cuda.cu
src/ggml-cuda/argsort.cu
src/ggml-cuda/common.cuh
src/ggml-cuda/mmq.cuh
src/ggml-cuda/softmax.cu
src/ggml-cuda/vecdotq.cuh