]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
whisper : restore decoder temperature fallbacks
authorGeorgi Gerganov <redacted>
Sat, 15 Apr 2023 13:04:07 +0000 (16:04 +0300)
committerGeorgi Gerganov <redacted>
Sat, 15 Apr 2023 13:12:55 +0000 (16:12 +0300)
commitf19e23fbd108ec3ac458c7a19b31c930719e7a94
tree0b5ea8f8a088d3919cf596aa583a070542493c4b
parentea1f8a50d4f70b54d9dd03205207a80019e243f9
whisper : restore decoder temperature fallbacks

I disabled this because there were many complaints about slow decoding.
The current implementation does not allow batching the decoders when
using the "best of" or "beam size" parameters, so the decoding time is
proportional to the number of decoders, which is obviously not great.

However, now there are even more complaints about wrong decodings and
repetition.

So, making a compromise by re-enabling the fallbacks, but defaulting to
just 2 "best of" / "beam size" decoders. Also, the temperature step is
increased from 0.2 to 0.4 - i.e. from maximum of 5 fallbacks to maximum
of 2.

Also, the stream example now has fallbacks enabled by default.

close #471 #477 #508 #612 #719 #731
examples/main/main.cpp
examples/stream/stream.cpp
whisper.cpp