From: Pierre Alexandre SCHEMBRI <redacted>
Date: Sat, 7 Oct 2023 10:29:33 +0000 (+0200)
Subject: readme : mention Metal could be used for gpt-2 (#553)
X-Git-Tag: upstream/0.0.1642~1229
X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=754ea2bd7a2bb8d9378a1c06bfe0cdc09f38126c;p=pkg%2Fggml%2Fsources%2Fggml

readme : mention Metal could be used for gpt-2 (#553)
---

diff --git a/README.md b/README.md
index ebb26296..5cfc8ba0 100644
--- a/README.md
+++ b/README.md
@@ -105,6 +105,19 @@ The inference speeds that I get for the different models on my 32GB MacBook M1 P
 
 For more information, checkout the corresponding programs in the [examples](examples) folder.
 
+## Using Metal (only with GPT-2)
+
+For GPT-2 models, offloading to GPU is possible. Note that it will not improve inference performances but will reduce power consumption and free up the CPU for other tasks.
+
+To enable GPU offloading on MacOS:
+
+```bash
+cmake -DGGML_METAL=ON -DBUILD_SHARED_LIBS=Off ..
+
+# add -ngl 1
+./bin/gpt-2 -t 4 -ngl 100 -m models/gpt-2-117M/ggml-model.bin -p "This is an example"
+```
+
 ## Using cuBLAS
 
 ```bash