]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
vulkan: Optimize contiguous copies (#10254)
authorJeff Bolz <redacted>
Wed, 13 Nov 2024 06:58:57 +0000 (00:58 -0600)
committerGitHub <redacted>
Wed, 13 Nov 2024 06:58:57 +0000 (07:58 +0100)
commit80dd7ff22fd050fed58b552cc8001aaf968b7ebf
treef134f6626bb07632baa88dfa292d45f5d6ca4aa8
parent54ef9cfc726a799e6f454ac22c4815d037716eda
vulkan: Optimize contiguous copies (#10254)

* tests: Fix memory bandwidth calculation for perf tests

Add a flops calculation for flash attention.

Add one GGML_OP_CPY perf test.

* vulkan: Optimize contiguous copies

Add a variant of the copy shader for when the tensors are contiguous. Avoid
the complex addressing calculations, and do four elements per invocation
to hide some other overhead.

Apply similar changes to the scale shader, since scale is always contiguous.

Add a "progress bar" for shader compiles.
13 files changed:
ggml/src/ggml-vulkan.cpp
ggml/src/vulkan-shaders/clamp.comp
ggml/src/vulkan-shaders/contig_copy.comp [new file with mode: 0644]
ggml/src/vulkan-shaders/copy.comp
ggml/src/vulkan-shaders/cos.comp
ggml/src/vulkan-shaders/generic_unary_head.comp
ggml/src/vulkan-shaders/pad.comp
ggml/src/vulkan-shaders/repeat.comp
ggml/src/vulkan-shaders/scale.comp
ggml/src/vulkan-shaders/sin.comp
ggml/src/vulkan-shaders/square.comp
ggml/src/vulkan-shaders/vulkan-shaders-gen.cpp
tests/test-backend-ops.cpp