]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
ggml : add CLBlast support (#1164)
author0cc4m <redacted>
Fri, 28 Apr 2023 14:57:16 +0000 (16:57 +0200)
committerGitHub <redacted>
Fri, 28 Apr 2023 14:57:16 +0000 (17:57 +0300)
commit7296c961d9303010a2b98379f738da2a8a55aa1b
tree398b36fb53bfab4411572cb69f861bbdbdbc2672
parent78ec543733d10a1629f984fd0302fdaa4e87fe66
ggml : add CLBlast support (#1164)

* Allow use of OpenCL GPU-based BLAS using ClBlast instead of OpenBLAS for context processing

* Improve ClBlast implementation, avoid recreating buffers, remove redundant transfers

* Finish merge of ClBlast support

* Move CLBlast implementation to separate file

Add buffer reuse code (adapted from slaren's cuda implementation)

* Add q4_2 and q4_3 CLBlast support, improve code

* Double CLBlast speed by disabling OpenBLAS thread workaround

Co-authored-by: Concedo <redacted>
Co-authored-by: slaren <redacted>
* Fix device selection env variable names

* Fix cast in opencl kernels

* Add CLBlast to CMakeLists.txt

* Replace buffer pool with static buffers a, b, qb, c

Fix compile warnings

* Fix typos, use GGML_TYPE defines, improve code

* Improve btype dequant kernel selection code, add error if type is unsupported

* Improve code quality

* Move internal stuff out of header
* Use internal enums instead of CLBlast enums
* Remove leftover C++ includes and defines
* Make event use easier to read

Co-authored-by: Henri Vasserman <redacted>
* Use c compiler for opencl files

* Simplify code, fix include

* First check error, then release event

* Make globals static, fix indentation

* Rename dequant kernels file to conform with other file names

* Fix import cl file name

---------

Co-authored-by: Concedo <redacted>
Co-authored-by: slaren <redacted>
Co-authored-by: Henri Vasserman <redacted>
Co-authored-by: Georgi Gerganov <redacted>
CMakeLists.txt
Makefile
ggml-opencl-dequant.cl [new file with mode: 0644]
ggml-opencl.c [new file with mode: 0644]
ggml-opencl.h [new file with mode: 0644]
ggml.c
ggml.h
llama.cpp