]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
threading: support for GGML_SCHED_PRIO_LOW, update thread info on Windows to avoid...
authorMax Krasnyansky <redacted>
Sat, 31 May 2025 22:39:19 +0000 (15:39 -0700)
committerGeorgi Gerganov <redacted>
Sun, 1 Jun 2025 11:01:05 +0000 (14:01 +0300)
commitbb57c3985c319539a2396c564e859c22b3ce26bd
tree93027fee6fa473a8649e9da45c20f85df0d63f42
parent078a221fc20160967d5133129d561b2b474e1977
threading: support for GGML_SCHED_PRIO_LOW, update thread info on Windows to avoid throttling (llama/12995)

* threading: support for GGML_SCHED_PRIO_LOW, update thread info on Windows to avoid throttling

We talked about adding LOW priority for GGML threads in the original threadpool PR.
It might be useful for some cases to avoid contention.

Latest Windows ARM64 releases started parking (offlining) the CPU cores
more aggresively which results in suboptimal performance with n_threads > 4.
To deal with that we now disable Power Throttling for our threads for the NORMAL
and higher priorities.

Co-authored-by: Diego Devesa <redacted>
* threading: disable SetThreadInfo() calls for older Windows versions

* Update tools/llama-bench/llama-bench.cpp

Co-authored-by: Diego Devesa <redacted>
---------

Co-authored-by: Diego Devesa <redacted>
include/ggml.h
src/ggml-cpu/ggml-cpu.c