]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
server : simplify state machine for slot (#9283)
authorXuan Son Nguyen <redacted>
Fri, 6 Sep 2024 21:21:29 +0000 (23:21 +0200)
committerGitHub <redacted>
Fri, 6 Sep 2024 21:21:29 +0000 (23:21 +0200)
commit9b2c24c0993487d3b34a873980e292da571481f3
tree0ece60a6ef16dc67cb7ca1c64d6dae07dec9723a
parent134bc38ecf3e2c5460581badce289a1ffa680453
server : simplify state machine for slot (#9283)

* server : simplify state machine for slot

* add SLOT_STATE_DONE_PROMPT

* pop_deferred_task

* add missing notify_one

* fix passkey test

* metrics : add n_busy_slots_per_decode

* fix test step

* add test

* maybe fix AddressSanitizer?

* fix deque ?

* missing lock

* pop_deferred_task: also notify

* Update examples/server/server.cpp

Co-authored-by: Georgi Gerganov <redacted>
---------

Co-authored-by: Georgi Gerganov <redacted>
examples/server/server.cpp
examples/server/tests/features/parallel.feature
examples/server/tests/features/passkey.feature
examples/server/tests/features/steps/steps.py