]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
overview / pkg / ggml / sources / llama.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: e1470ee)
server: (router) add LRU scheduler (#26572)
authorXuan-Son Nguyen <redacted>
Fri, 7 Aug 2026 12:46:53 +0000 (14:46 +0200)
committerGitHub <redacted>
Fri, 7 Aug 2026 12:46:53 +0000 (14:46 +0200)
commitdff15d4ac95de1a3c73e8b784d4c436f5e5e36eb
tree58e72fbe0de027a1be4af1d85dde0d9898d59bb6tree
parente1470ee6a2a9a44a3246c7b832efd1756badbb9ecommit | diff
server: (router) add LRU scheduler (#26572)

* add lru_sched

* handle coalescing (req leaves waiting queue)

* add tests

* fix stream case

* address review comments
tools/server/server-models.cpp diff | blob | history
tools/server/server-models.h diff | blob | history
tools/server/tests/unit/test_router.py diff | blob | history
tools/server/tests/utils.py diff | blob | history
Packaging of ggml-org/llama.cpp