]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
vulkan: optimizations for direct convolution (#14933)
authorJeff Bolz <redacted>
Sat, 2 Aug 2025 07:57:04 +0000 (02:57 -0500)
committerGitHub <redacted>
Sat, 2 Aug 2025 07:57:04 +0000 (09:57 +0200)
commita9f7541ec25c4c8547daf5ff48700ad2836e2b7d
tree4574ffb3b0f22606e7149e3e7f98f109a4d76ffd
parent9c35706b98ea271858acef4194f526a71b24cdc9
vulkan: optimizations for direct convolution (#14933)

* vulkan: optimizations for direct convolution

- Empirically choose a better tile size. Reducing BS_K/BS_NPQ helps fill
  the GPU. The new size should be amenable to using coopmat, too.
- Fix shmem bank conflicts. 16B padding should work with coopmat.
- Some explicit loop unrolling.
- Skip math/stores work for parts of the tile that are OOB.
- Apply fastdiv opt.
- Disable shuffles for NV.

* Three tiles sizes for CONV_2D, and a heuristic to choose

* reallow collectives for pre-Turing

* make SHMEM_PAD a spec constant

* fixes for intel perf - no shmem padding, placeholder shader core count

* shader variants with/without unrolling

* 0cc4m's fixes for AMD perf

Co-authored-by: 0cc4m <redacted>
---------

Co-authored-by: 0cc4m <redacted>
ggml/src/ggml-vulkan/ggml-vulkan.cpp
ggml/src/ggml-vulkan/vulkan-shaders/conv2d_mm.comp
ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp