]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
overview / pkg / ggml / sources / llama.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 971f245)
CANN: Add support for async operator submission (#12864)
authorhipudding <redacted>
Thu, 17 Apr 2025 12:34:16 +0000 (20:34 +0800)
committerGitHub <redacted>
Thu, 17 Apr 2025 12:34:16 +0000 (20:34 +0800)
commit7a395f67a7a02bb361d944b816d6e933889e28e1
tree58442de89b02986700e6234beb9049c2552329dbtree
parent971f245b3b5f3f55991bb779cb541b00f82eea1dcommit | diff
CANN: Add support for async operator submission (#12864)

Submit operators using asynchronous threads to improve performance.

Use the environment variable GGML_CANN_ASYNC_MODE to control whether
asynchronous submission is enabled. It is disabled by default.

Testing shows a 10%–20% performance improvement in scenarios with
small parameter sizes, especially in quantized models.
ggml/src/ggml-cann/aclnn_ops.cpp diff | blob | history
ggml/src/ggml-cann/aclnn_ops.h diff | blob | history
ggml/src/ggml-cann/common.h diff | blob | history
ggml/src/ggml-cann/ggml-cann.cpp diff | blob | history
Packaging of ggml-org/llama.cpp