]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
overview / pkg / ggml / sources / ggml / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 8386c72)
ggml : IQ4_NL sgemm + Q4_0 AVX optimization (llama/9422)
authorEve <redacted>
Mon, 16 Sep 2024 06:48:24 +0000 (06:48 +0000)
committerGeorgi Gerganov <redacted>
Fri, 20 Sep 2024 19:03:57 +0000 (22:03 +0300)
commitdf55df2c324562ed3fbceeb3ff00e7620daecaf5
tree303049da9526e85edd631e2b6d0203d08007aa1etree
parent8386c72901144d1b45addde752791fe3f255e6c5commit | diff
ggml : IQ4_NL sgemm + Q4_0 AVX optimization (llama/9422)

* squashed

readd my iq4_nl sgemm PR https://github.com/ggerganov/llama.cpp/pull/8049

have ggml_vec_dot_q4_0 do two blocks per loop for avx

try out f16c ggml_vec_dot_iq4_nl, but it's not really faster. as per https://github.com/ggerganov/llama.cpp/pull/8549 we can calculate several blocks at a time with no issue

* shuffle

* remove f16c iq4_nl as i cant make it faster than before
src/ggml-quants.c diff | blob | history
Packaging of ggml-org/ggml