]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
overview / pkg / ggml / sources / ggml / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 3e0c111)
rpc : cache and reuse compute graphs (llama/15405)
authorRadoslav Gerganov <redacted>
Fri, 28 Nov 2025 08:33:51 +0000 (10:33 +0200)
committerGeorgi Gerganov <redacted>
Thu, 11 Dec 2025 13:32:48 +0000 (15:32 +0200)
commit910f59e765464eda2863c37648acc18a0ae0ba0e
tree5e58091e2a9a5abfad78b143122fda0ded653333tree
parent3e0c1113ef4ea0654303c9ba19365f3c568c082fcommit | diff
rpc : cache and reuse compute graphs (llama/15405)

Store the last computed graph and reuse it when possible.
Also do not return response from GRAPH_COMPUTE and assume it always
completes successfully. If this this is not the case, the server closes
the connection. This saves us a network round trip to the server.
include/ggml-rpc.h diff | blob | history
src/ggml-rpc/ggml-rpc.cpp diff | blob | history
Packaging of ggml-org/ggml