]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
overview / pkg / ggml / sources / llama.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: e475fa2)
server : optimize get_token_probabilities (#24796)
authorAdrien Gallouët <redacted>
Fri, 19 Jun 2026 21:26:54 +0000 (23:26 +0200)
committerGitHub <redacted>
Fri, 19 Jun 2026 21:26:54 +0000 (23:26 +0200)
commit4b48a53b6cc60e051f35f2acbd06264a909bb255
tree6f676d741722b57f044497b5259c7b1cf5813920tree
parente475fa2b5f9fb50c3d6fc3e7c6fdf1e004465b62commit | diff
server : optimize get_token_probabilities (#24796)

Use std::partial_sort to order only the requested top-n tokens instead
of the full vocabulary

    logprobs sort: vocab=128000 n_top=0 iters=100
    full    sort:   8555.6 us/op
    partial sort:    704.3 us/op

Signed-off-by: Adrien Gallouët <redacted>
tools/server/server-common.cpp diff | blob | history
tools/server/server-common.h diff | blob | history
tools/server/server-context.cpp diff | blob | history
Packaging of ggml-org/llama.cpp