]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
sampling : refactor + optimize penalties sampler (#10803)
authorGeorgi Gerganov <redacted>
Mon, 16 Dec 2024 10:31:14 +0000 (12:31 +0200)
committerGitHub <redacted>
Mon, 16 Dec 2024 10:31:14 +0000 (12:31 +0200)
commit644fd71b44c4cdbfc6482fbf0353d289c3bc29e6
tree1a47aa420dda05600627900bd65292485967d01a
parent4ddd199f6f6b980e0a7ed9f9b44efeae2fbdf5c4
sampling : refactor + optimize penalties sampler (#10803)

* sampling : refactor + optimize penalties sampler

ggml-ci

* common : apply ignore_eos as logit bias

ggml-ci

* batched : remove penalties sampler

* params : allow penalty_last_n == -1 to be equal to context size

ggml-ci

* common : by default, move the penalties at the end of the sampling chain

ggml-ci

* common : ignore all EOG tokens

Co-authored-by: Diego Devesa <redacted>
* common : move back the penalties at the front of the sampling chain

ggml-ci

* readme : restore hint about --ignore-eos flag [no ci]

* llama : minor

ggml-ci

* webui : update

---------

Co-authored-by: Diego Devesa <redacted>
17 files changed:
common/arg.cpp
common/common.cpp
common/common.h
common/sampling.cpp
examples/batched/batched.cpp
examples/main/README.md
examples/server/README.md
examples/server/public/index.html.gz
examples/server/public_legacy/index-new.html
examples/server/public_legacy/index.html
examples/server/server.cpp
examples/server/themes/buttons-top/index.html
examples/server/themes/wild/index.html
examples/server/webui/src/main.js
include/llama.h
src/llama-sampling.cpp
tests/test-sampling.cpp