]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
vulkan: Optimize contiguous copies (llama/10254)
authorJeff Bolz <redacted>
Wed, 13 Nov 2024 06:58:57 +0000 (00:58 -0600)
committerGeorgi Gerganov <redacted>
Fri, 15 Nov 2024 13:21:04 +0000 (15:21 +0200)
commit21b01a21b6d5b6d3527c8901c8ad471821b30b97
tree7e3fa88993faeeafe208e6a1a57e7a53b2654ec7
parentb54ce5edc5125da1e4ce8050dbb86330d7b0f9b6
vulkan: Optimize contiguous copies (llama/10254)

* tests: Fix memory bandwidth calculation for perf tests

Add a flops calculation for flash attention.

Add one GGML_OP_CPY perf test.

* vulkan: Optimize contiguous copies

Add a variant of the copy shader for when the tensors are contiguous. Avoid
the complex addressing calculations, and do four elements per invocation
to hide some other overhead.

Apply similar changes to the scale shader, since scale is always contiguous.

Add a "progress bar" for shader compiles.
12 files changed:
ggml/src/ggml-vulkan.cpp
ggml/src/vulkan-shaders/clamp.comp
ggml/src/vulkan-shaders/contig_copy.comp [new file with mode: 0644]
ggml/src/vulkan-shaders/copy.comp
ggml/src/vulkan-shaders/cos.comp
ggml/src/vulkan-shaders/generic_unary_head.comp
ggml/src/vulkan-shaders/pad.comp
ggml/src/vulkan-shaders/repeat.comp
ggml/src/vulkan-shaders/scale.comp
ggml/src/vulkan-shaders/sin.comp
ggml/src/vulkan-shaders/square.comp
ggml/src/vulkan-shaders/vulkan-shaders-gen.cpp