]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
CPU/CUDA: Gemma 2 FlashAttention support (llama/8542)
authorJohannes Gäßler <redacted>
Sat, 24 Aug 2024 19:34:59 +0000 (21:34 +0200)
committerGeorgi Gerganov <redacted>
Tue, 27 Aug 2024 19:01:14 +0000 (22:01 +0300)
commit141bc623c354579bb6abd9507ac5cbc1165efb64
treeae268d400a131f01f449f02a8dd727d2a07eceac
parentcd8374c1bd992a97761bffe6a3e271bb24a167f0
CPU/CUDA: Gemma 2 FlashAttention support (llama/8542)

* CPU/CUDA: Gemma 2 FlashAttention support

* apply logit_softcap to scale in kernel

* disable logit softcapping tests on Metal

* remove metal check
include/ggml.h
src/ggml-cuda/fattn-common.cuh
src/ggml-cuda/fattn-tile-f16.cu
src/ggml-cuda/fattn-tile-f32.cu
src/ggml-cuda/fattn-vec-f16.cuh
src/ggml-cuda/fattn-vec-f32.cuh
src/ggml-cuda/fattn-wmma-f16.cuh
src/ggml-cuda/fattn.cu
src/ggml-metal.m
src/ggml.c
tests/test-backend-ops.cpp