]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
CANN: Optimize CANN buffer pool memory management (llama/12875)
authorXinpeng Dou <redacted>
Tue, 15 Apr 2025 02:04:24 +0000 (10:04 +0800)
committerGeorgi Gerganov <redacted>
Thu, 24 Apr 2025 15:36:25 +0000 (18:36 +0300)
commit039f1765e667ebbe2b6f40e9bdbdcf3a58d08907
tree87854013f6fe2e1f7c71ba601a30f7e6bc776050
parent8b75c002fe44f74632d24944a39154b40bc1d348
CANN: Optimize CANN buffer pool memory management (llama/12875)

Multiple optional memory pools are provided for CANN, including VMM,
priority queue-based, and traditional memory pools.
1.When the memory pool is available and GGML_CANN_DISABLE_VMM_POOL
   is not defined, the VMM pool is selected by default.
2.Otherwise, if GGML_CANN_ENABLE_BUF_PRIO_POOL is defined,
   the priority queue-based memory pool is used.
3.If neither condition is met, the default memory pool is used.
src/ggml-cann/aclnn_ops.cpp
src/ggml-cann/ggml-cann.cpp