]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
ggml: Add POOL2D OP for GPU acceleration to the Vulkan backend in the MobileVLM model...
authorChangyeon Kim <redacted>
Tue, 29 Oct 2024 08:52:56 +0000 (17:52 +0900)
committerGeorgi Gerganov <redacted>
Mon, 4 Nov 2024 17:42:09 +0000 (19:42 +0200)
commit9e4f8907a9612494fdfc8da06978c334d6dfde8b
tree376c128229bede7e9301a7520e93795a0f683fbf
parent5db70fabf1279d16dd360d994eee7781e71836e7
ggml: Add POOL2D OP for GPU acceleration to the Vulkan backend in the MobileVLM model. (llama/9763)

* ggml: Add POOL2D OP for GPU ACC to the Vulkan.

- The MobileVLM model now supports inference acceleration through GPU by utilizing the Vulkan backend.
- A GGML_OP_POOL_2D shader has been added. (Pooling)
- The encoding performance of the CLIP model improved from 2.8s on the CPU to 0.7s on the GPU.

Signed-off-by: Changyeon Kim <redacted>
* [fix] Correct the incorrect order of the parameters.

fix casting to int.

Signed-off-by: Changyeon Kim <redacted>
---------

Signed-off-by: Changyeon Kim <redacted>
src/ggml-vulkan.cpp
src/vulkan-shaders/pool2d.comp [new file with mode: 0644]
src/vulkan-shaders/vulkan-shaders-gen.cpp