]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
overview / pkg / ggml / sources / llama.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 50a8a2a)
ggml : add new Q4_2 quantization (ARM only) (#1046)
authorGeorgi Gerganov <redacted>
Tue, 18 Apr 2023 20:54:57 +0000 (23:54 +0300)
committerGitHub <redacted>
Tue, 18 Apr 2023 20:54:57 +0000 (23:54 +0300)
commit77a73403ca8eaced2590559d0f9cebd2b3649d32
tree7b95e7565ce86b81d8dd620117564da901ce3ce7tree
parent50a8a2af97cb92e53e7a3195aa201c3d87da5415commit | diff
ggml : add new Q4_2 quantization (ARM only) (#1046)

* ggml : Q4_2 ARM

* ggml : add ggml_is_quantized()

* llama : update llama_type_name() with Q4_2 entry

* ggml : speed-up q4_2

- 4 threads: ~100ms -> ~90ms
- 8 threads:  ~55ms -> ~50ms

* ggml : optimize q4_2 using vmlaq_n_f32 + vmulq_n_f32
examples/quantize/quantize.cpp diff | blob | history
ggml.c diff | blob | history
ggml.h diff | blob | history
llama.cpp diff | blob | history
llama.h diff | blob | history
Packaging of ggml-org/llama.cpp