From: Mirko185 Date: Mon, 19 Feb 2024 07:39:31 +0000 (+0100) Subject: readme : update (#5572) X-Git-Tag: upstream/0.0.4488~2299 X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=769a716e30ba1da46f709df1c00727d6869d30e7;p=pkg%2Fggml%2Fsources%2Fllama.cpp readme : update (#5572) Added 1.5-bit on README.md --- diff --git a/README.md b/README.md index 8c7bc268..70866e24 100644 --- a/README.md +++ b/README.md @@ -61,7 +61,7 @@ variety of hardware - locally and in the cloud. - Plain C/C++ implementation without any dependencies - Apple silicon is a first-class citizen - optimized via ARM NEON, Accelerate and Metal frameworks - AVX, AVX2 and AVX512 support for x86 architectures -- 2-bit, 3-bit, 4-bit, 5-bit, 6-bit, and 8-bit integer quantization for faster inference and reduced memory use +- 1.5-bit, 2-bit, 3-bit, 4-bit, 5-bit, 6-bit, and 8-bit integer quantization for faster inference and reduced memory use - Custom CUDA kernels for running LLMs on NVIDIA GPUs (support for AMD GPUs via HIP) - Vulkan, SYCL, and (partial) OpenCL backend support - CPU+GPU hybrid inference to partially accelerate models larger than the total VRAM capacity