]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
overview / pkg / ggml / sources / ggml / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 4151b04)
whisper : suppress tokens with a regex (whisper/1997)
authorulatekh <redacted>
Tue, 9 Apr 2024 15:27:28 +0000 (08:27 -0700)
committerGeorgi Gerganov <redacted>
Tue, 9 Apr 2024 17:28:26 +0000 (20:28 +0300)
commit7607e21a511afa6aeb51fd86bf0b18c2e8c34246
tree64453a06a379b0de7bfa6e661f4316649f51dc6atree
parent4151b0420d7a8f4c3c1b420afa0f62ca441b9cd8commit | diff
whisper : suppress tokens with a regex (whisper/1997)

* Allow a regular expression to describe tokens to suppress.

Example: --suppress-tokens-re "[,\.]|[ ]?[0-9]+" will suppress commas, periods, and numeric tokens.

Technique inspired by https://github.com/openai/whisper/discussions/1041

Co-authored-by: Georgi Gerganov <redacted>
* Blind change to fix Java test.

---------

Co-authored-by: Georgi Gerganov <redacted>
examples/whisper/main.cpp diff | blob | history
examples/whisper/whisper.cpp diff | blob | history
examples/whisper/whisper.h diff | blob | history
Packaging of ggml-org/ggml