]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
overview / pkg / ggml / sources / llama.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 413e7b0)
server : refactored the task processing logic (#5065)
authorXuan Son Nguyen <redacted>
Fri, 26 Jan 2024 12:42:20 +0000 (13:42 +0100)
committerGitHub <redacted>
Fri, 26 Jan 2024 12:42:20 +0000 (14:42 +0200)
commit48c857aa10aea73210a4a72da3f1a6f99269e75d
tree84ad3c5956c03c1bac8922f68ca828e456116b50tree
parent413e7b0559f922bd4de5e9eec548829d111651b1commit | diff
server : refactored the task processing logic (#5065)

* server: add llama_server_queue struct

* server: add llama_server_response_event

* server: add comments

* server: move all mutexes away from server.cpp

* server: correct multitask response

* server: only add back deferred tasks when one slot is available

* server: fix a race condition cause by "request_completion"
Makefile diff | blob | history
examples/server/CMakeLists.txt diff | blob | history
examples/server/oai.hpp [new file with mode: 0644] blob
examples/server/server.cpp diff | blob | history
examples/server/utils.hpp [new file with mode: 0644] blob
Packaging of ggml-org/llama.cpp