]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
speculative : add tree-based sampling example (#3624)
authorGeorgi Gerganov <redacted>
Wed, 18 Oct 2023 13:21:57 +0000 (16:21 +0300)
committerGitHub <redacted>
Wed, 18 Oct 2023 13:21:57 +0000 (16:21 +0300)
commit0e89203b517c95ec6675eda75d200a60d1e8921d
tree3aba40ef0362d061f240bd43c52e86a8f728f89d
parentc67fe68e417f766970fb1feaf2e66458aa24116a
speculative : add tree-based sampling example (#3624)

* sampling : one sequence per sampling context

ggml-ci

* speculative : add tree-based sampling support

ggml-ci

* speculative : reuse the n_parallel CLI param

* speculative : refactor sampling

* examples : fix build after sampling refactoring

ggml-ci

* batched : fix n_seq_id

* sampling : fix malloc

ggml-ci

* swift : fix build

ggml-ci

* swift : try to fix build

ggml-ci

* prompts : add assistant.txt

* common : add llama_batch_add() and llama_batch_clear() helpers

* speculative : minor refactor

ggml-ci

* minor : comments + rename

ggml-ci

* speculative : fix off-by-one for n_drafted

* speculative : fix the n_drafted fix + p constants
21 files changed:
Makefile
common/common.cpp
common/common.h
common/log.h
common/sampling.cpp
common/sampling.h
examples/batched-bench/batched-bench.cpp
examples/batched.swift/Sources/main.swift
examples/batched/batched.cpp
examples/embd-input/embd-input-lib.cpp
examples/infill/infill.cpp
examples/llava/llava-utils.h
examples/llava/llava.cpp
examples/main/main.cpp
examples/parallel/parallel.cpp
examples/server/server.cpp
examples/simple/simple.cpp
examples/speculative/speculative.cpp
llama.cpp
llama.h
prompts/assistant.txt [new file with mode: 0644]