]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
overview / pkg / ggml / sources / llama.cpp / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 96013c5)
ui: reduce per-token render cost when streaming (#26053)
authorZach Winter <redacted>
Fri, 24 Jul 2026 20:09:46 +0000 (16:09 -0400)
committerGitHub <redacted>
Fri, 24 Jul 2026 20:09:46 +0000 (22:09 +0200)
commit555881ebc8b0fc0402b30e09258a32a7bfd13c52
tree8c0b2e13c271baf98c1e159771ed63ac8471b70atree
parent96013c51121fcabbe00b70095f104962abe1ec14commit | diff
ui: reduce per-token render cost when streaming  (#26053)

* performance harness - the empirical root

Assisted-by: Claude Opus 4.8
* 210.36ms -> 2.67ms per streamed token

Assisted-by: Claude Opus 4.8
* 11.58ms -> 0.62ms per streamed token

Assisted-by: Claude Opus 4.8
* 22.02ms -> 3.33ms per streamed token

Assisted-by: Claude Opus 4.8
* 3.07ms -> 1.36ms per streamed token at 40 messages

Assisted-by: Claude Opus 4.8
---------

Co-authored-by: Zach Winter <redacted>
18 files changed:
tools/ui/src/lib/components/app/content/CollapsibleContentBlock.svelte diff | blob | history
tools/ui/src/lib/components/app/content/CollapsibleTerminalBlock.svelte diff | blob | history
tools/ui/src/lib/components/app/content/MarkdownContent/MarkdownContent.svelte diff | blob | history
tools/ui/src/lib/constants/latex-protection.ts diff | blob | history
tools/ui/src/lib/stores/conversations.svelte.ts diff | blob | history
tools/ui/src/lib/utils/code.ts diff | blob | history
tools/ui/src/lib/utils/latex-protection.ts diff | blob | history
tools/ui/tests/client/README-perf.md [new file with mode: 0644] blob
tools/ui/tests/client/agentic-stream.perf.svelte.test.ts [new file with mode: 0644] blob
tools/ui/tests/client/collapsible-lazy-body.svelte.test.ts [new file with mode: 0644] blob
tools/ui/tests/client/components/AgenticPerfWrapper.svelte [new file with mode: 0644] blob
tools/ui/tests/client/components/ChatMessagesPerfWrapper.svelte [new file with mode: 0644] blob
tools/ui/tests/client/components/CollapsibleLazyBodyHarness.svelte [new file with mode: 0644] blob
tools/ui/tests/client/components/agentic-perf-state.svelte.ts [new file with mode: 0644] blob
tools/ui/tests/client/update-message-in-place.svelte.test.ts [new file with mode: 0644] blob
tools/ui/tests/unit/agentic-hotpath.bench.ts [new file with mode: 0644] blob
tools/ui/tests/unit/code.test.ts diff | blob | history
tools/ui/tests/unit/latex-protection.test.ts diff | blob | history
Packaging of ggml-org/llama.cpp