]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
SYCL: Add set_rows support for quantized types (#14883)
authorAkarshan Biswas <redacted>
Mon, 28 Jul 2025 15:02:15 +0000 (20:32 +0530)
committerGitHub <redacted>
Mon, 28 Jul 2025 15:02:15 +0000 (20:32 +0530)
commitcd1fce6d4f9c191f1c7429cc96f61281c3b63ffc
treedae0b4252d84a6ae2cab6cd965f8bf1b5d5d09f9
parent00fa15fedc79263fa0285e6a3bbb0cfb3e3878a2
SYCL: Add set_rows support for quantized types  (#14883)

* SYCL: Add set_rows support for quantized types

This commit adds support for GGML_OP_SET_ROWS operation for various
quantized tensor types (Q8_0, Q5_1, Q5_0, Q4_1, Q4_0, IQ4_NL) and BF16
type in the SYCL backend.

The quantization/dequantization copy kernels were moved from cpy.cpp
to cpy.hpp to make them available for set_rows.cpp.

This addresses part of the TODOs mentioned in the code.

* Use get_global_linear_id() instead

ggml-ci

* Fix formatting

ggml-ci

* Use const for ne11 and size_t variables in set_rows_sycl_q

ggml-ci

* Increase block size for q kernel to 256

ggml-ci

* Cleanup imports

* Add float.h to cpy.hpp
ggml/src/ggml-sycl/cpy.cpp
ggml/src/ggml-sycl/cpy.hpp
ggml/src/ggml-sycl/ggml-sycl.cpp
ggml/src/ggml-sycl/set_rows.cpp