]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
convert : ignore tokens if their IDs are within [0, vocab_size) (#3831)
authorGeorgi Gerganov <redacted>
Sat, 28 Oct 2023 12:25:15 +0000 (15:25 +0300)
committerGitHub <redacted>
Sat, 28 Oct 2023 12:25:15 +0000 (06:25 -0600)
commit8a2f2fea2914aaa3f4b2f82800c7de15f15bdb09
tree5ec6d747892f61952219fcd3e2b8f82bcc897add
parentbd6d9e205982b34e0ba2c3b22bbf31a1ef1a1bb5
convert : ignore tokens if their IDs are within [0, vocab_size) (#3831)
convert.py