]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
overview / pkg / ggml / sources / ggml / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: ad2754e)
starcoder : add <|end_of_turn|> token handling in order to support openchat/opencoder...
authorthe-crypt-keeper <redacted>
Mon, 10 Jul 2023 18:41:58 +0000 (14:41 -0400)
committerGitHub <redacted>
Mon, 10 Jul 2023 18:41:58 +0000 (21:41 +0300)
commitd40d7ff57bd1ac27eec1a3bf1ae46e428a038252
tree7b377f3410ba7d531b30d7c8e840829f52e27d3dtree
parentad2754ef9fe64c80d8baa63b5a3ad362c8ea227acommit | diff
starcoder : add <|end_of_turn|> token handling in order to support openchat/opencoderplus (#343)

* Add <|end_of_turn|> token handling to support openchat/opencoderplus

* The opencoder EOT occurs inside the prompt, so we should only break if the model actually generated it

---------

Co-authored-by: Mike <redacted>
examples/starcoder/main.cpp diff | blob | history
Packaging of ggml-org/ggml