From: Kylin <redacted>
Date: Tue, 22 Aug 2023 07:14:23 +0000 (+0800)
Subject: ggml : support CUDA's half type for aarch64(#1455) (#2670)
X-Git-Tag: gguf-v0.4.0~289
X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=1e3bc523d8053a77df3ac7126a84d0297ee97ef6;p=pkg%2Fggml%2Fsources%2Fllama.cpp

ggml : support CUDA's half type for aarch64(#1455) (#2670)

* ggml: support CUDA's half type for aarch64(#1455)
support CUDA's half type for aarch64 in ggml_fp16_t definition

* ggml: use __CUDACC__ to recognise nvcc compiler
---

diff --git a/ggml.h b/ggml.h
index 544ad2d1..0ec7ec5b 100644
--- a/ggml.h
+++ b/ggml.h
@@ -259,8 +259,9 @@
 extern "C" {
 #endif
 
-#ifdef __ARM_NEON
-    // we use the built-in 16-bit float type
+#if defined(__ARM_NEON) && defined(__CUDACC__)
+    typedef half ggml_fp16_t;
+#elif defined(__ARM_NEON)
     typedef __fp16 ggml_fp16_t;
 #else
     typedef uint16_t ggml_fp16_t;