]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
hexagon: minor refresh for HMX FA and MM (llama/23796)
authorMax Krasnyansky <redacted>
Thu, 28 May 2026 11:49:11 +0000 (04:49 -0700)
committerGeorgi Gerganov <redacted>
Fri, 29 May 2026 06:47:30 +0000 (09:47 +0300)
commit1b241b879c4687d7ff4b3af1a14cb8e491a70d2d
treeb4fef77b5e8edcc87d023bed2cef716b56c9344d
parentb896e91f18ec245f1415fe5d18a77e766197985e
hexagon: minor refresh for HMX FA and MM (llama/23796)

* hex-fa: clean up qf32/fp32 handling and stride handling

* hex-fa: fix corner case fp NAN issues that were cause bad output from gemma4 on v79

* hex-fa: vectorize leftover handling

* hex-fa: avoid HVX fallback during token gen HMX has more FP16 compute capacity

* hmx-mm: remove dead code

* hmx-mm: use fastdiv in x4x2 dequant

* hmx-mm: sandwich dequant and scatter to improve perf

* hmx-mm: fixed rebase conflicts

* hmx-mm: further improve weight dequant by doing early type dispatch and precomputing fastdiv

* hmx-mm: an even earlier dispatch for per-type dequant

* hmx-mm: dequant linear types like q4_0 and q4_1 without the LUTs

This is a bit faster than LUT.

* hex-cmake: one more tweak for lto

---------

Co-authored-by: Trivikram Reddy <redacted>
ggml/src/ggml-hexagon/htp/CMakeLists.txt
ggml/src/ggml-hexagon/htp/flash-attn-ops.c
ggml/src/ggml-hexagon/htp/hmx-flash-attn-ops.c
ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c