]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
overview / pkg / ggml / sources / llama.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 71ca2fa)
speculative : add heuristic algorithm (#3006)
authorLeng Yue <redacted>
Thu, 14 Sep 2023 16:14:44 +0000 (09:14 -0700)
committerGitHub <redacted>
Thu, 14 Sep 2023 16:14:44 +0000 (19:14 +0300)
commit35f73049af6c676a106a5a990a819ae0bc3fcd7d
tree55807c47e621aca6ffe3cb8936ade0f3f80e2921tree
parent71ca2fad7d6c0ef95ef9944fb3a1a843e481f314commit | diff
speculative : add heuristic algorithm (#3006)

* Add heuristic algo for speculative

* Constrain minimum n_draft to 2

* speculative : improve heuristic impl

* speculative : be more rewarding upon guessing max drafted tokens

* speculative : fix typos

---------

Co-authored-by: Georgi Gerganov <redacted>
examples/speculative/speculative.cpp diff | blob | history
Packaging of ggml-org/llama.cpp