]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
overview / pkg / ggml / sources / whisper.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 33bdbfb)
rpc : do not wait for response when sending RPC_CMD_SET_TENSOR (llama/12943)
authorRadoslav Gerganov <redacted>
Fri, 25 Apr 2025 07:08:08 +0000 (10:08 +0300)
committerGeorgi Gerganov <redacted>
Thu, 1 May 2025 10:29:02 +0000 (13:29 +0300)
commitfe21ddf0dcaf4af68694b8cae8608278266be20c
treec528d250ed7912490a13a330d282ef7446b2781ftree
parent33bdbfbb33568e6adb311292fc456444c8d40b99commit | diff
rpc : do not wait for response when sending RPC_CMD_SET_TENSOR (llama/12943)

RPC_CMD_SET_TENSOR always returns an empty response and we send this 4
times per token. We can improve TG speed if we don't wait for this empty
response.

The performance impact of this change depends on the network latency.
ggml/include/ggml-rpc.h diff | blob | history
ggml/src/ggml-rpc/ggml-rpc.cpp diff | blob | history
Packaging of ggerganov/whisper.cpp