]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
server : add single-client multi-prompt support (#4232)
authorZiad Ben Hadj-Alouane <redacted>
Thu, 30 Nov 2023 22:25:04 +0000 (17:25 -0500)
committerGitHub <redacted>
Thu, 30 Nov 2023 22:25:04 +0000 (00:25 +0200)
commitf43f09366dfd018e4568e23a232aaa8c4f7cfc78
tree5ebe4b191f1201991aa5656aeacbc9b9d23da05e
parentd2809a3ba2780e00fce5a6149a7eda09f1c0e906
server : add single-client multi-prompt support (#4232)

* * add multiprompt support

* * cleanup

* * more cleanup

* * remove atomicity of id_gen, and change lock_guard to unique_lock on completion requests

* * remove all references to mutex_multitasks

* Update examples/server/server.cpp

Co-authored-by: Jared Van Bortel <redacted>
* Update examples/server/server.cpp

Co-authored-by: Jared Van Bortel <redacted>
* Update examples/server/server.cpp

Co-authored-by: Jared Van Bortel <redacted>
* Update examples/server/server.cpp

Co-authored-by: Jared Van Bortel <redacted>
* * change to set

---------

Co-authored-by: Jared Van Bortel <redacted>
examples/server/server.cpp