]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
llama : optimize long word tokenization with WPM (#8034)
authorGeorgi Gerganov <redacted>
Fri, 21 Jun 2024 05:51:28 +0000 (08:51 +0300)
committerGitHub <redacted>
Fri, 21 Jun 2024 05:51:28 +0000 (08:51 +0300)
commita927b0f3dd9a86ee042cd2bdcc8c9da4a855926b
treecb718eba3db4e581a0814d3f0b6a952b7a3218b1
parent80ea089d771f0c2d97afa8bead80ded412f600d7
llama : optimize long word tokenization with WPM (#8034)

ggml-ci
llama.cpp
unicode.cpp