From: matteo <redacted>
Date: Thu, 1 Aug 2024 21:28:28 +0000 (+0200)
Subject: ggml-cuda: Adding support for unified memory (llama/8035)
X-Git-Tag: upstream/0.0.1642~473
X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=0edb8d80fc57cd79c61914f814ee7a9979e654f0;p=pkg%2Fggml%2Fsources%2Fggml

ggml-cuda: Adding support for unified memory (llama/8035)

* Adding support for unified memory

* adding again the documentation about unified memory

* refactoring: Moved the unified memory code in the correct location.

* Fixed compilation error when using hipblas

* cleaning up the documentation

* Updating the documentation

Co-authored-by: Johannes Gäßler <redacted>

* adding one more case where the PR should not be enabled

---------

Co-authored-by: matteo serva <redacted>
Co-authored-by: Johannes Gäßler <redacted>
---

diff --git a/src/ggml-cuda.cu b/src/ggml-cuda.cu
index b510777f..68605fff 100644
--- a/src/ggml-cuda.cu
+++ b/src/ggml-cuda.cu
@@ -130,7 +130,22 @@ static cudaError_t ggml_cuda_device_malloc(void ** ptr, size_t size, int device)
     }
     return res;
 #else
+
+#if !defined(GGML_USE_HIPBLAS) && !defined(GGML_USE_MUSA)
+    cudaError_t err;
+    if (getenv("GGML_CUDA_ENABLE_UNIFIED_MEMORY") != nullptr)
+    {
+        err = cudaMallocManaged(ptr, size);
+    }
+    else
+    {
+        err = cudaMalloc(ptr, size);
+    }
+    return err;
+#else
     return cudaMalloc(ptr, size);
+#endif // !defined(GGML_USE_HIPBLAS) && !defined(GGML_USE_MUSA)
+
 #endif
 }