]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
metal : extend mul_mv_ext to BF16, Q2_K, Q3_K (llama/20250)
authorPaul Flynn <redacted>
Mon, 9 Mar 2026 14:48:12 +0000 (10:48 -0400)
committerGeorgi Gerganov <redacted>
Mon, 16 Mar 2026 11:10:15 +0000 (13:10 +0200)
commitae21974f4f3ded7c06f97804527009612a0be948
treea3780ab7bd0831d75cd2555d5449ee40286a8d30
parentd19c65e9daa7eb0ea383cf666aed9e3580c4a844
metal : extend mul_mv_ext to BF16, Q2_K, Q3_K (llama/20250)

Enable mul_mv_ext small-batch kernels (BS 2-8) for BF16, Q2_K,
and Q3_K quantization types. These types previously fell through
to the slower single-row mul_mv path.

BF16 uses the float4 dequantize path (like F16). Q2_K and Q3_K
use the float4x4 K-quant path (like Q4_K/Q5_K/Q6_K).

Co-authored-by: Claude Opus 4.6 <redacted>
ggml/src/ggml-metal/ggml-metal-ops.cpp
ggml/src/ggml-metal/ggml-metal.metal