]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
speculative : refactor and add a simpler example (#10362)
authorGeorgi Gerganov <redacted>
Mon, 25 Nov 2024 07:58:41 +0000 (09:58 +0200)
committerGitHub <redacted>
Mon, 25 Nov 2024 07:58:41 +0000 (09:58 +0200)
commitd9d54e498d38ec99bbc0031022f9c92711e97bbc
tree24159c07d784c1926f80e3b9fc9a950d56da2048
parentcce5a9007572c6e9fa522296b77571d2e5071357
speculative : refactor and add a simpler example (#10362)

* speculative : refactor and add a simpler example

ggml-ci

* speculative : clean-up and add comments and TODOs [no ci]

* speculative : manage context in common_speculative

ggml-ci

* speculative : simplify

ggml-ci

* speculative : simplify (cont)

ggml-ci

* speculative : add --draft-min CLI arg

* speculative : minor fixup

* make : build fixes

* speculative : do not redraft previous drafts

ggml-ci

* speculative : fix the draft sampling

ggml-ci

* speculative : fix compile warning

* common : refactor args

ggml-ci

* common : change defaults [no ci]

* common : final touches

ggml-ci
28 files changed:
Makefile
common/CMakeLists.txt
common/arg.cpp
common/common.cpp
common/common.h
common/sampling.cpp
common/sampling.h
common/speculative.cpp [new file with mode: 0644]
common/speculative.h [new file with mode: 0644]
examples/CMakeLists.txt
examples/batched/batched.cpp
examples/infill/infill.cpp
examples/llava/llava-cli.cpp
examples/llava/minicpmv-cli.cpp
examples/lookahead/lookahead.cpp
examples/lookup/lookup-stats.cpp
examples/lookup/lookup.cpp
examples/main/main.cpp
examples/parallel/parallel.cpp
examples/retrieval/retrieval.cpp
examples/save-load-state/save-load-state.cpp
examples/server/server.cpp
examples/server/utils.hpp
examples/speculative-simple/CMakeLists.txt [new file with mode: 0644]
examples/speculative-simple/README.md [new file with mode: 0644]
examples/speculative-simple/speculative-simple.cpp [new file with mode: 0644]
examples/speculative/speculative.cpp
tests/test-arg-parser.cpp