]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
hexagon: optimization for HMX mat_mul (llama/21554)
authorYiwei Shao <redacted>
Tue, 14 Apr 2026 21:09:03 +0000 (14:09 -0700)
committerGeorgi Gerganov <redacted>
Thu, 30 Apr 2026 08:29:09 +0000 (11:29 +0300)
commit24cc89e477bea0336e911316d036c24dee5258a8
tree18b8adf88ce680e95490aeecb21a6de04e127504
parent44d86c4921c1d8ba48e946c1885983311e6055b1
hexagon: optimization for HMX mat_mul (llama/21554)

* hexagon: add async HMX worker

Introduce hmx-worker (dedicated thread for HMX compute) to overlap HMX
matmul with HVX dequant/DMA stages in the pipeline path, replacing the
previous synchronous HMX calls that blocked the main thread.

* hexagon: cost-based VTCM chunk search for out-stationary matmul

* hexagon: fix futex race in hmx_worker_drain
Store the boolean to local variable avoid atomic load twice

* hex-mm: hmx optimize scatter/transpose and use HMX intrinsics

* hex-vmem: drop vmem limit a touch under 3GB on v73

* hexagon: add fwd declaration of htp_context

* hex-hmx: replace hmx-worker with hmx-queue that mimics dma-queue interface

Simplifies the overall implemantion, reduces thread wakeup roundtrips.

* hex-mm: add debug log to hmx work func called from hmx-queue

* Update hmx-queue.h

Co-authored-by: Max Krasnyansky <redacted>
---------

Co-authored-by: Kim-Chyan Gan <redacted>
Co-authored-by: Max Krasnyansky <redacted>
Co-authored-by: Max Krasnyansky <redacted>
ggml/src/ggml-hexagon/htp/CMakeLists.txt
ggml/src/ggml-hexagon/htp/hex-utils.h
ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c
ggml/src/ggml-hexagon/htp/hmx-queue.c [new file with mode: 0644]
ggml/src/ggml-hexagon/htp/hmx-queue.h [new file with mode: 0644]
ggml/src/ggml-hexagon/htp/hmx-utils.h
ggml/src/ggml-hexagon/htp/htp-ctx.h
ggml/src/ggml-hexagon/htp/htp-ops.h
ggml/src/ggml-hexagon/htp/hvx-base.h
ggml/src/ggml-hexagon/htp/main.c