]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
sampling : avoid expensive softmax during greedy sampling (#9605)
authorGeorgi Gerganov <redacted>
Tue, 24 Sep 2024 06:03:17 +0000 (09:03 +0300)
committerGitHub <redacted>
Tue, 24 Sep 2024 06:03:17 +0000 (09:03 +0300)
commitb0f27361f3539a81d983a8b045f3c61e682d9fc0
tree7d1ef5055499d53ff0cd24ed005df09dedaf3e61
parentc087b6f11d3385f4293b6841ebfb755063479490
sampling : avoid expensive softmax during greedy sampling (#9605)

* sampling : avoid expensive softmax during greedy sampling

ggml-ci

* speculative : fix default RNG seed + set sparams.n_probs

* Update tests/test-sampling.cpp

Co-authored-by: slaren <redacted>
* sampling : add clarifying comment [no ci]

---------

Co-authored-by: slaren <redacted>
common/sampling.cpp
examples/speculative/speculative.cpp
include/llama.h
src/llama-sampling.cpp
tests/test-sampling.cpp