From: Sam Spilsbury <redacted>
Date: Wed, 9 Aug 2023 20:47:42 +0000 (+0300)
Subject: ggml-alloc: Don't try to re-use buffers of external tensors (#2562)
X-Git-Tag: gguf-v0.4.0~334
X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=916a9acdd0a411426690400ebe2bb7ce840a6bba;p=pkg%2Fggml%2Fsources%2Fllama.cpp

ggml-alloc: Don't try to re-use buffers of external tensors (#2562)

* ggml-alloc: Don't try to re-use buffers of external tensors

They might be weights that came from another context, so we
have no control over them (and they might be re-used elsewhere
so writing to them would be a bad idea).

* ggml-alloc: >= when checking for out-of-bounds

Co-authored-by: slaren <redacted>

---------

Co-authored-by: slaren <redacted>
---

diff --git a/ggml-alloc.c b/ggml-alloc.c
index 5e1be61f..4121f3db 100644
--- a/ggml-alloc.c
+++ b/ggml-alloc.c
@@ -394,6 +394,14 @@ static void allocate_node(struct ggml_allocr * alloc, struct ggml_tensor * node)
                     if (parent == NULL) {
                         break;
                     }
+
+                    // if the node's data is external, then we cannot re-use it
+                    if ((char *) parent->data < (char *) alloc->data ||
+                        (char *) parent->data >= ((char *) alloc->data + alloc->size)) {
+                        AT_PRINTF("not reusing parent %s for %s as %p is external\n", parent->name, node->name, parent->data);
+                        continue;
+                    }
+
                     struct hash_node * p_hn = hash_get(ht, parent);
                     if (parent->data != NULL && p_hn->n_children == 1 && p_hn->n_views == 0 && ggml_are_same_layout(node, parent)) {
                         if (ggml_is_view(parent)) {