]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
run : avoid double tokenization (#14327)
authorRuikai Peng <redacted>
Sun, 22 Jun 2025 17:28:06 +0000 (01:28 +0800)
committerGitHub <redacted>
Sun, 22 Jun 2025 17:28:06 +0000 (01:28 +0800)
commit66aba7aca9a245d71f1ddf02c7a97223529752a8
tree99a083c523e853f6de480f4cb8a194ce3554f1f8
parentf1f5e82df6222dcbaca6396c0de44df259a1694f
run : avoid double tokenization (#14327)

* run : avoid double tokenization by adopting common_tokenize heuristic

* build : fix windows gcc and clang warnings

* lint : fixed trailing whitepace

* run : fix is_first flag
tools/run/run.cpp