]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
SYCL: Add set_rows support for quantized types (llama/14883)
authorAkarshan Biswas <redacted>
Mon, 28 Jul 2025 15:02:15 +0000 (20:32 +0530)
committerGeorgi Gerganov <redacted>
Sat, 2 Aug 2025 14:51:21 +0000 (17:51 +0300)
commitb3bc1c7635eca47cc168d93dd483bc674d1b05ad
tree4c774ec6545ff214e5ee5c1ad47dd9a36df63d04
parent5ba3f90c70d34618a77cac20f3b06d921987a647
SYCL: Add set_rows support for quantized types (llama/14883)

* SYCL: Add set_rows support for quantized types

This commit adds support for GGML_OP_SET_ROWS operation for various
quantized tensor types (Q8_0, Q5_1, Q5_0, Q4_1, Q4_0, IQ4_NL) and BF16
type in the SYCL backend.

The quantization/dequantization copy kernels were moved from cpy.cpp
to cpy.hpp to make them available for set_rows.cpp.

This addresses part of the TODOs mentioned in the code.

* Use get_global_linear_id() instead

ggml-ci

* Fix formatting

ggml-ci

* Use const for ne11 and size_t variables in set_rows_sycl_q

ggml-ci

* Increase block size for q kernel to 256

ggml-ci

* Cleanup imports

* Add float.h to cpy.hpp
src/ggml-sycl/cpy.cpp
src/ggml-sycl/cpy.hpp
src/ggml-sycl/ggml-sycl.cpp
src/ggml-sycl/set_rows.cpp