]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
threading: support for GGML_SCHED_PRIO_LOW, update thread info on Windows to avoid...
authorMax Krasnyansky <redacted>
Sat, 31 May 2025 22:39:19 +0000 (15:39 -0700)
committerGitHub <redacted>
Sat, 31 May 2025 22:39:19 +0000 (15:39 -0700)
commit053b1539c02617eff744f89525ee57497c3c1fbe
tree901c2d1362779612f27f71e3eef928641885473f
parentb3a89c3d9e34c28c5be70d8b687a84775746d4a0
threading: support for GGML_SCHED_PRIO_LOW, update thread info on Windows to avoid throttling (#12995)

* threading: support for GGML_SCHED_PRIO_LOW, update thread info on Windows to avoid throttling

We talked about adding LOW priority for GGML threads in the original threadpool PR.
It might be useful for some cases to avoid contention.

Latest Windows ARM64 releases started parking (offlining) the CPU cores
more aggresively which results in suboptimal performance with n_threads > 4.
To deal with that we now disable Power Throttling for our threads for the NORMAL
and higher priorities.

Co-authored-by: Diego Devesa <redacted>
* threading: disable SetThreadInfo() calls for older Windows versions

* Update tools/llama-bench/llama-bench.cpp

Co-authored-by: Diego Devesa <redacted>
---------

Co-authored-by: Diego Devesa <redacted>
common/arg.cpp
common/common.cpp
ggml/include/ggml.h
ggml/src/ggml-cpu/ggml-cpu.c
tools/llama-bench/llama-bench.cpp