From: Michael de Gans Date: Thu, 20 Jun 2024 05:32:01 +0000 (-0700) Subject: metal : fix `ggml_metal_supports_op` for BF16 (#8021) X-Git-Tag: upstream/0.0.4488~1300 X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=2075a66a96cc1b04eabec7cf4b3051193d6f719e;p=pkg%2Fggml%2Fsources%2Fllama.cpp metal : fix `ggml_metal_supports_op` for BF16 (#8021) Currently the Metal backend does not support BF16. `ggml_metal_supports_op` was returning true in these cases, leading to a crash with models converted with `--leave-output-tensor`. This commit checks if the first few sources types are BF16 and returns false if that's the case. --- diff --git a/ggml-metal.m b/ggml-metal.m index f894274c..79902c9a 100644 --- a/ggml-metal.m +++ b/ggml-metal.m @@ -735,6 +735,12 @@ static id ggml_metal_get_buffer(struct ggml_tensor * t, size_t * offs } static bool ggml_metal_supports_op(const struct ggml_metal_context * ctx, const struct ggml_tensor * op) { + for (size_t i = 0, n = 3; i < n; ++i) { + if (op->src[i] != NULL && op->src[i]->type == GGML_TYPE_BF16) { + return false; + } + } + switch (op->op) { case GGML_OP_UNARY: switch (ggml_get_unary_op(op)) {