]>
git.djapps.eu Git - pkg/ggml/sources/ggml/commit
Add ability to use Q5_0, Q5_1, and IQ4_NL for quantized K cache (llama/6183)
* k_cache: be able to use Q5_0
* k_cache: be able to use Q5_1 on CODA
* k_cache: be able to use Q5_0 on Metal
* k_cache: be able to use Q5_1 on Metal
* k_cache: be able to use IQ4_NL - just CUDA for now
* k_cache: be able to use IQ4_NL on Metal
* k_cache: add newly added supported types to llama-bench and CUDA supports_op
---------
Co-authored-by: Iwan Kawrakow <redacted>
overview / pkg / ggml / sources / ggml / commit