]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
ggml : add RISC-V Vector Support for K-Quants and improved the existing intrinsics...
authorTameem <redacted>
Tue, 3 Oct 2023 18:38:19 +0000 (23:38 +0500)
committerGitHub <redacted>
Tue, 3 Oct 2023 18:38:19 +0000 (21:38 +0300)
commit79f34abddb72ac5ddbf118f3d87520b611a10a7d
treea3c46ea79eebaa697d902370cf2c295910166f39
parent8186242b6d67cf87ae179fb1a62f52fdf0e5c5eb
ggml : add RISC-V Vector Support for K-Quants and improved the existing intrinsics (#3453)

* Added RVV intrinsics support for Q8 quantize row and also improved the existing dot product function for risc-v.

The RVV intrinsics is added for the following quantize row functions
   quantize_row_q8_0
   quantize_row_q8_1

The following dot product functions have also been optimized by using LMUL = 1/2 instead of LMUL = 1
   ggml_vec_dot_q4_0_q8_0
   ggml_vec_dot_q4_1_q8_1
   ggml_vec_dot_q5_0_q8_0
   ggml_vec_dot_q5_1_q8_1

And vector initialization in Q5 by temporary array is also replaced by the vid intrinsics

Signed-off-by: Ahmad Tameem <redacted>
* Added RVV intrinsics support for k_quants

This adds RISC-V Vector intrinsics support for the following K_quants functions for both QKK = 256 and QKK = 64
   ggml_vec_dot_q2_K_q8_K
   ggml_vec_dot_q3_K_q8_K
   ggml_vec_dot_q4_K_q8_K
   ggml_vec_dot_q5_K_q8_K
   ggml_vec_dot_q6_K_q8_K

Signed-off-by: Ahmad Tameem <redacted>
---------

Signed-off-by: Ahmad Tameem <redacted>
ggml.c
k_quants.c