]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
llama: limit max outputs of `llama_context` (#23861)
authorAman Gupta <redacted>
Mon, 1 Jun 2026 15:01:38 +0000 (23:01 +0800)
committerGitHub <redacted>
Mon, 1 Jun 2026 15:01:38 +0000 (18:01 +0300)
commitde6f727aaec7dc477629946d80c803a0bb7af0a1
tree030cae757438b46dbc4223cd48f7511924473ee8
parent95b8b8ec1a9e77bcc7b3fb04da82c9f35cb12a79
llama: limit max outputs of `llama_context` (#23861)

* llama: save more VRAM by reserving n_outputs == n_seqs when possible

* add n_outputs_per_seq

* move n_outputs_max to server-context

* change ubatch to batch everywhere
common/common.cpp
common/common.h
include/llama.h
src/llama-context.cpp
src/llama-cparams.h
tools/server/server-context.cpp