]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
SYCL: Add set_rows support for quantized types (llama/14883)
authorAkarshan Biswas <redacted>
Mon, 28 Jul 2025 15:02:15 +0000 (20:32 +0530)
committerGeorgi Gerganov <redacted>
Mon, 18 Aug 2025 17:30:45 +0000 (20:30 +0300)
commit4908e9dd0513e4174fcca24a5f09b8509a591115
tree511c366b55ad69c24c3a0e961bd27c728a433959
parent24d3524bfd147dcd3d3f440da4966cf7f1c85d31
SYCL: Add set_rows support for quantized types (llama/14883)

* SYCL: Add set_rows support for quantized types

This commit adds support for GGML_OP_SET_ROWS operation for various
quantized tensor types (Q8_0, Q5_1, Q5_0, Q4_1, Q4_0, IQ4_NL) and BF16
type in the SYCL backend.

The quantization/dequantization copy kernels were moved from cpy.cpp
to cpy.hpp to make them available for set_rows.cpp.

This addresses part of the TODOs mentioned in the code.

* Use get_global_linear_id() instead

ggml-ci

* Fix formatting

ggml-ci

* Use const for ne11 and size_t variables in set_rows_sycl_q

ggml-ci

* Increase block size for q kernel to 256

ggml-ci

* Cleanup imports

* Add float.h to cpy.hpp
ggml/src/ggml-sycl/cpy.cpp
ggml/src/ggml-sycl/cpy.hpp
ggml/src/ggml-sycl/ggml-sycl.cpp
ggml/src/ggml-sycl/set_rows.cpp