]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
server: refactor + correctness fixes for metrics (#26920)
authorXuan-Son Nguyen <redacted>
Thu, 13 Aug 2026 08:02:01 +0000 (10:02 +0200)
committerGitHub <redacted>
Thu, 13 Aug 2026 08:02:01 +0000 (10:02 +0200)
commitdecaf508bb9ef683c93f58d320b6d6faef507895
tree8f241254660898d0660455a2beb50accf7cebd10
parente79e4bf660e19f2ad851e06c6913f7a8c5852621
server: refactor + correctness fixes for metrics (#26920)

* server: refactor metrics

* move most fields to server_slot_stats

* cont

* rm result_timings

* tie stats to batch

* cont

* nits: move place in code

* exclude first generated token

* more accurate batch metrics tracking

* n_predict --> n_gen

* metrics_on_prediction

* metrics_flush_idle

* metrics: seperate cache/processed prompt tokens

* refactor server_task_result_metrics

* add test

* nits

* fix flush before reset()

* cont

* rm dead code

* nits
tools/server/server-common.cpp
tools/server/server-common.h
tools/server/server-context.cpp
tools/server/server-task.cpp
tools/server/server-task.h
tools/server/tests/unit/test_metrics.py [new file with mode: 0644]