]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
lookup : add prompt lookup decoding example (#4484)
authorLeonEricsson <redacted>
Fri, 22 Dec 2023 16:05:56 +0000 (17:05 +0100)
committerGitHub <redacted>
Fri, 22 Dec 2023 16:05:56 +0000 (18:05 +0200)
commit7082d24cec35e9ce9147535a2224dfc67ee0a78c
treeb87d0e65d71c8e2a5bdb889483c75d4429d2d566
parentba661751322a7c201fd3bef71af077c5aebfaa2a
lookup : add prompt lookup decoding example (#4484)

* initial commit, going through initializations

* main loop finished, starting to debug

* BUG: generates gibberish/repeating tokens after a while

* kv_cache management

* Added colors to distinguish drafted tokens (--color). Updated README

* lookup : fix token positions in the draft batch

* lookup : use n_draft from CLI params

* lookup : final touches

---------

Co-authored-by: Leon Ericsson <redacted>
Co-authored-by: Georgi Gerganov <redacted>
.gitignore
Makefile
common/common.h
examples/CMakeLists.txt
examples/lookup/CMakeLists.txt [new file with mode: 0644]
examples/lookup/README.md [new file with mode: 0644]
examples/lookup/lookup.cpp [new file with mode: 0644]