]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
overview / pkg / ggml / sources / whisper.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 1fe5948)
CUDA: use tensor cores for MMQ (llama/7676)
authorJohannes Gäßler <redacted>
Mon, 10 Jun 2024 09:45:13 +0000 (11:45 +0200)
committerGeorgi Gerganov <redacted>
Sun, 16 Jun 2024 15:19:48 +0000 (18:19 +0300)
commit7483d2b61cd534e01c433ae059519fd0b909b50a
tree5b928ddb27ded5c9acf1dde968811901e514896atree
parent1fe594822728e3c655ba3f0d66156cbdbbd33177commit | diff
CUDA: use tensor cores for MMQ (llama/7676)

* CUDA: int8 tensor cores for MMQ (legacy quants)

* fix out-of-bounds writes

* __builtin_assume -> GGML_CUDA_ASSUME

* fix writeback returning too early
ggml-cuda/common.cuh diff | blob | history
ggml-cuda/fattn-common.cuh diff | blob | history
ggml-cuda/fattn-tile-f16.cu diff | blob | history
ggml-cuda/fattn-vec-f16.cuh diff | blob | history
ggml-cuda/fattn-wmma-f16.cuh diff | blob | history
ggml-cuda/mma.cuh [new file with mode: 0644] blob
ggml-cuda/mmq.cuh diff | blob | history
Packaging of ggerganov/whisper.cpp