From: Julius Tischbein <redacted>
Date: Wed, 15 Oct 2025 11:54:15 +0000 (+0200)
Subject: CUDA: Changing the CUDA scheduling strategy to spin (#16585)
X-Git-Tag: upstream/0.0.7011~244
X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=5acd455460f457942d8dd02e3dd9b1eebfce99fe;p=pkg%2Fggml%2Fsources%2Fllama.cpp

CUDA: Changing the CUDA scheduling strategy to spin (#16585)

* CUDA set scheduling strategy to spinning for cc121

* Using prop.major and prop.minor, include HIP and MUSA

* Exclude HIP and MUSA

* Remove trailing whitespace

Co-authored-by: Johannes Gäßler <redacted>

* Remove empty line

Co-authored-by: Johannes Gäßler <redacted>

---------

Co-authored-by: Johannes Gäßler <redacted>
---

diff --git a/ggml/src/ggml-cuda/ggml-cuda.cu b/ggml/src/ggml-cuda/ggml-cuda.cu
index da312992..a5e77672 100644
--- a/ggml/src/ggml-cuda/ggml-cuda.cu
+++ b/ggml/src/ggml-cuda/ggml-cuda.cu
@@ -273,6 +273,15 @@ static ggml_cuda_device_info ggml_cuda_init() {
         } else if (device_name.substr(0, 21) == "NVIDIA GeForce GTX 16") {
             turing_devices_without_mma.push_back({ id, device_name });
         }
+
+        // Temporary performance fix:
+        // Setting device scheduling strategy for iGPUs with cc121 to "spinning" to avoid delays in cuda synchronize calls.
+        // TODO: Check for future drivers the default scheduling strategy and
+        // remove this call again when cudaDeviceScheduleSpin is default.
+        if (prop.major == 12 && prop.minor == 1) {
+            CUDA_CHECK(cudaSetDeviceFlags(cudaDeviceScheduleSpin));
+        }
+
 #endif  // defined(GGML_USE_HIP)
     }