]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
llava: Add ACC OP for GPU acceleration to the Vulkan backend in the LLAVA CLIP model...
authorChangyeon Kim <redacted>
Tue, 20 Aug 2024 19:00:00 +0000 (04:00 +0900)
committerGitHub <redacted>
Tue, 20 Aug 2024 19:00:00 +0000 (21:00 +0200)
commit2f3c1466ff46a2413b0e363a5005c46538186ee6
tree580ec571b2bf403eb3982f2ae3d8203201fc4105
parent50addec9a532a6518146ab837a85504850627316
llava: Add ACC OP for GPU acceleration to the Vulkan backend in the LLAVA CLIP model. (#8984)

* llava: Add ACC OP for GPU acceleration to the Vulkan backend in the LLAVA CLIP model.

- The CLIP model now prioritizes the Vulkan backend over the CPU when vulkan available.
- A GGML_OP_ACC shader has been added.
- The encoding performance of the CLIP model improved from 4.2s on the CPU to 0.9s on the GPU.

Signed-off-by: Changyeon Kim <redacted>
* fix-up coding style.

Signed-off-by: Changyeon Kim <redacted>
* Fix-up the missing initial parameter to resolve the compilation warning.

Signed-off-by: Changyeon Kim <redacted>
* [fix] Add missing parameters.

Signed-off-by: Changyeon Kim <redacted>
* [fix] Use nb1 and nb2 for dst.

Signed-off-by: Changyeon Kim <redacted>
* Fix check results ggml_acc call

---------

Signed-off-by: Changyeon Kim <redacted>
Co-authored-by: 0cc4m <redacted>
examples/llava/clip.cpp
ggml/src/ggml-vulkan.cpp
ggml/src/vulkan-shaders/acc.comp [new file with mode: 0644]
ggml/src/vulkan-shaders/vulkan-shaders-gen.cpp