From: Georgi Gerganov Date: Thu, 20 Apr 2023 20:35:52 +0000 (+0300) Subject: stablelm : update README.md X-Git-Tag: upstream/0.0.1642~1534 X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=05f307971862b83df12fada0c42ee027ba5a82b5;p=pkg%2Fggml%2Fsources%2Fggml stablelm : update README.md --- diff --git a/examples/stablelm/README.md b/examples/stablelm/README.md index 7882d8b3..7a70ca72 100644 --- a/examples/stablelm/README.md +++ b/examples/stablelm/README.md @@ -56,9 +56,9 @@ main: predict time = 4474.07 ms / 63.92 ms per token main: total time = 6911.26 ms ``` -## Run in 4-bit integer quantization mode +## 4-bit integer quantization mode -``` +```bash # quantize the model to 4-bits using Q4_3 quantization ./bin/stablelm-quantize ./stablelm-base-alpha-3b/ggml-model-f16.bin ./stablelm-base-alpha-3b/ggml-model-q4_3.bin 6