]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
server : fix speculative decoding with context shift (#10641)
authorGeorgi Gerganov <redacted>
Wed, 4 Dec 2024 20:38:20 +0000 (22:38 +0200)
committerGitHub <redacted>
Wed, 4 Dec 2024 20:38:20 +0000 (22:38 +0200)
commit1da7b765692764a8b33b08da61cbee63812a7bd9
tree05f7991c07a3230f5c51d34f908d16bf1aeaf9ea
parent59f4db10883a4f3e855cffbf2c3ab68430e95272
server : fix speculative decoding with context shift (#10641)

* server : fix speculative decoding with context shift

ggml-ci

* server : take into account speculative limits

ggml-ci

* server : add tests
examples/server/server.cpp
examples/server/tests/unit/test_speculative.py