]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
hexagon: dma optimizations (mostly fixing regressions) (llama/21137)
authorMax Krasnyansky <redacted>
Sun, 29 Mar 2026 13:40:13 +0000 (06:40 -0700)
committerGeorgi Gerganov <redacted>
Wed, 1 Apr 2026 13:00:26 +0000 (16:00 +0300)
commit06754dd1de5a94ee9161fa88b1e7117d832d07d3
tree7996809ef3ba41c3d7ada3681029d4ce27b5ba70
parenta04eea0761a85d18f3f504d6ab970c5c9dce705f
hexagon: dma optimizations (mostly fixing regressions) (llama/21137)

* hex-fa: add simple dma cache for Mask

I noticed that we were refetch the mask rows over and over.
This simple cache avoids that.

* hex-dma: unset in-order desc bit which caused signficant perf regression

We don't rely on true in order processing of the DMA descriptors anywhere.
Turns out this mode caused significant regression of around 3-4 TPS during token gen.

* hex-rope: update comment to clarify that we don't need in-order DMA completions
src/ggml-hexagon/htp/flash-attn-ops.c
src/ggml-hexagon/htp/hex-dma.h
src/ggml-hexagon/htp/rope-ops.c