]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
Optimize mul_mat for Q4_0 on Intel GPU (llama/12035)
authorNeo Zhang Jianyu <redacted>
Mon, 24 Feb 2025 14:33:23 +0000 (22:33 +0800)
committerGeorgi Gerganov <redacted>
Thu, 27 Feb 2025 06:55:36 +0000 (08:55 +0200)
commite3cb412a59fcc33d5543d66fc2146b5c1e126cb9
tree57744445ac0d760a2a58311e1c4a4f937fbffed2
parentac301a7d9b4ca4dad3a3b7de37a29dcaeeca9347
Optimize mul_mat for Q4_0 on Intel GPU (llama/12035)

* opt performance by reorder for Intel GPU

* detect hw type and save opt feature, and print opt feature

* correct name

* support optimize graph once when compute graph, record the opt status in tensor->extra, make CI passed

* add env variable GGML_SYCL_DISABLE_OPT for debug

* use syclex::architecture replace the custom hw define, update the guide for GGML_SYCL_DISABLE_OPT

* add performance data

* mv getrows functions to separeted files

* fix global variables

---------

Co-authored-by: arthw <redacted>
12 files changed:
ggml/src/ggml-sycl/CMakeLists.txt
ggml/src/ggml-sycl/common.cpp
ggml/src/ggml-sycl/common.hpp
ggml/src/ggml-sycl/convert.cpp
ggml/src/ggml-sycl/convert.hpp
ggml/src/ggml-sycl/dequantize.hpp
ggml/src/ggml-sycl/dmmv.cpp
ggml/src/ggml-sycl/getrows.cpp [new file with mode: 0644]
ggml/src/ggml-sycl/getrows.hpp [new file with mode: 0644]
ggml/src/ggml-sycl/ggml-sycl.cpp
ggml/src/ggml-sycl/sycl_hw.cpp [new file with mode: 0644]
ggml/src/ggml-sycl/sycl_hw.hpp [new file with mode: 0644]