]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
vulkan: optimizations for direct convolution (llama/14933)
authorJeff Bolz <redacted>
Sat, 2 Aug 2025 07:57:04 +0000 (02:57 -0500)
committerGeorgi Gerganov <redacted>
Sat, 2 Aug 2025 14:51:21 +0000 (17:51 +0300)
commit0167c0972dc3a3374bb560a5339bf761c5abb4de
treecea65075cc87562f470ec0cdf2eb64e283f745c4
parent5b41763a4ec2c76ada457b7ee97450a028c44235
vulkan: optimizations for direct convolution (llama/14933)

* vulkan: optimizations for direct convolution

- Empirically choose a better tile size. Reducing BS_K/BS_NPQ helps fill
  the GPU. The new size should be amenable to using coopmat, too.
- Fix shmem bank conflicts. 16B padding should work with coopmat.
- Some explicit loop unrolling.
- Skip math/stores work for parts of the tile that are OOB.
- Apply fastdiv opt.
- Disable shuffles for NV.

* Three tiles sizes for CONV_2D, and a heuristic to choose

* reallow collectives for pre-Turing

* make SHMEM_PAD a spec constant

* fixes for intel perf - no shmem padding, placeholder shader core count

* shader variants with/without unrolling

* 0cc4m's fixes for AMD perf

Co-authored-by: 0cc4m <redacted>
---------

Co-authored-by: 0cc4m <redacted>
src/ggml-vulkan/ggml-vulkan.cpp
src/ggml-vulkan/vulkan-shaders/conv2d_mm.comp
src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp