]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
server : fix logprobs, make it OAI-compatible (#10783)
authorXuan Son Nguyen <redacted>
Thu, 19 Dec 2024 14:40:08 +0000 (15:40 +0100)
committerGitHub <redacted>
Thu, 19 Dec 2024 14:40:08 +0000 (15:40 +0100)
commit57bb2c40cd94c5a09f5210ed8264cc93b21c4b7e
treee07f347babeaa30f6f417a7f7622d825affec18d
parenta3c33b1dce2d4f25040b75f66629104bd1e40128
server : fix logprobs, make it OAI-compatible (#10783)

* server : fix logprobs, make it openai-compatible

* update docs

* add std::log

* return pre-sampling p

* sort before apply softmax

* add comment

* fix test

* set p for sampled token

* update docs

* add --multi-token-probs

* update docs

* add `post_sampling_probs` option

* update docs [no ci]

* remove --multi-token-probs

* "top_probs" with "post_sampling_probs"

* resolve review comments

* rename struct token_prob to prob_info

* correct comment placement

* fix setting prob for sampled token
examples/server/README.md
examples/server/server.cpp
examples/server/tests/unit/test_chat_completion.py
examples/server/tests/unit/test_completion.py
examples/server/tests/unit/test_embedding.py
examples/server/utils.hpp