]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
starcoder : add support for starchat special tokens (#246)
authorRavindra Marella <redacted>
Sun, 18 Jun 2023 07:37:09 +0000 (13:07 +0530)
committerGitHub <redacted>
Sun, 18 Jun 2023 07:37:09 +0000 (10:37 +0300)
commite456108433017d5586b35fd36ce781b4c3aed631
treed1d736ee62680011d5ad67c38154db944d7893e9
parent070f170ac03987ad888b3c37050185a859a3c011
starcoder : add support for starchat special tokens (#246)

* starcoder : add support for starchat special tokens

* examples : fix `gpt_tokenize()` for special tokens
examples/common.cpp
examples/common.h
examples/starcoder/main.cpp