]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
overview / pkg / ggml / sources / whisper.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: cb7642b)
CANN: Add support for async operator submission (llama/12864)
authorhipudding <redacted>
Thu, 17 Apr 2025 12:34:16 +0000 (20:34 +0800)
committerGeorgi Gerganov <redacted>
Thu, 24 Apr 2025 17:39:16 +0000 (20:39 +0300)
commit314ce5981e653158d3ed438a4eb7dd640f912eff
tree15b31bdac52a751e881aa77eb17bbabac833df93tree
parentcb7642b0f508d02a1992a1cf3784ce17990c4343commit | diff
CANN: Add support for async operator submission (llama/12864)

Submit operators using asynchronous threads to improve performance.

Use the environment variable GGML_CANN_ASYNC_MODE to control whether
asynchronous submission is enabled. It is disabled by default.

Testing shows a 10%–20% performance improvement in scenarios with
small parameter sizes, especially in quantized models.
ggml/src/ggml-cann/aclnn_ops.cpp diff | blob | history
ggml/src/ggml-cann/aclnn_ops.h diff | blob | history
ggml/src/ggml-cann/common.h diff | blob | history
ggml/src/ggml-cann/ggml-cann.cpp diff | blob | history
Packaging of ggerganov/whisper.cpp