]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
hexagon: optimization for HMX mat_mul (#21554)
authorYiwei Shao <redacted>
Tue, 14 Apr 2026 21:09:03 +0000 (14:09 -0700)
committerGitHub <redacted>
Tue, 14 Apr 2026 21:09:03 +0000 (14:09 -0700)
commit5d14e5d19bd6af7fc38eb92d96aa185e5948a03d
tree4438799782d523f270e1b7125d8a5d5343f8006b
parentfae3a28070fe4026f87bd6a544aba1b2d1896566
hexagon: optimization for HMX mat_mul (#21554)

* hexagon: add async HMX worker

Introduce hmx-worker (dedicated thread for HMX compute) to overlap HMX
matmul with HVX dequant/DMA stages in the pipeline path, replacing the
previous synchronous HMX calls that blocked the main thread.

* hexagon: cost-based VTCM chunk search for out-stationary matmul

* hexagon: fix futex race in hmx_worker_drain
Store the boolean to local variable avoid atomic load twice

* hex-mm: hmx optimize scatter/transpose and use HMX intrinsics

* hex-vmem: drop vmem limit a touch under 3GB on v73

* hexagon: add fwd declaration of htp_context

* hex-hmx: replace hmx-worker with hmx-queue that mimics dma-queue interface

Simplifies the overall implemantion, reduces thread wakeup roundtrips.

* hex-mm: add debug log to hmx work func called from hmx-queue

* Update hmx-queue.h

Co-authored-by: Max Krasnyansky <redacted>
---------

Co-authored-by: Kim-Chyan Gan <redacted>
Co-authored-by: Max Krasnyansky <redacted>
Co-authored-by: Max Krasnyansky <redacted>
ggml/src/ggml-hexagon/htp/CMakeLists.txt
ggml/src/ggml-hexagon/htp/hex-utils.h
ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c
ggml/src/ggml-hexagon/htp/hmx-queue.c [new file with mode: 0644]
ggml/src/ggml-hexagon/htp/hmx-queue.h [new file with mode: 0644]
ggml/src/ggml-hexagon/htp/hmx-utils.h
ggml/src/ggml-hexagon/htp/htp-ctx.h
ggml/src/ggml-hexagon/htp/htp-ops.h
ggml/src/ggml-hexagon/htp/hvx-base.h
ggml/src/ggml-hexagon/htp/main.c