]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
llama-bench: add --devices and --list-devices support (#16039)
authorssweens <redacted>
Fri, 19 Sep 2025 22:15:21 +0000 (15:15 -0700)
committerGitHub <redacted>
Fri, 19 Sep 2025 22:15:21 +0000 (00:15 +0200)
commitbe79d9fdd95ab8955527c4aaa67b90e8b9516718
tree77ab2475570e4f024b271f8315b49046e35a6fc2
parentf432d8d83e7407073634c5e4fd81a3d23a10827f
llama-bench: add --devices and --list-devices support (#16039)

* * llama-bench: add --devices support
- Support --devices same as llama-server
- Provide for benchmarking different device combinations
- Include --list-devices like llama-server for convenience

* fix: field display ordering restored

* fix: integrated the rpc devices
- aimed to mimic the server as much as possible

* cleanup: defaults for list-devices
- handle dup device listing with RPC

* cleanup: remove dup device load calls

* docs: update llama-bench
- added the recently added n-cpu-moe option to the docs while in there

* llama-bench: rpc device simplification
* rpc servers unify with other devices earlier, simplifying code
* --list-devices made stateless and simpler
* various cleanup
tools/llama-bench/README.md
tools/llama-bench/llama-bench.cpp