]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
vulkan: add TQ2_0 (ternary) support (llama/25850)
authormichaeltrabalka-tech <redacted>
Wed, 12 Aug 2026 05:07:23 +0000 (01:07 -0400)
committerGeorgi Gerganov <redacted>
Fri, 14 Aug 2026 19:16:06 +0000 (22:16 +0300)
commit632904002ddb0fa9f9731d287ada96fa5f54f11f
treeada78f1c460bc223a4b774224ce9ce023293198f
parentdb3687b2979a116a4556d8478e284f28080d8e6c
vulkan: add TQ2_0 (ternary) support (llama/25850)

* vulkan: TQ2_0 (ternary) support — dequant + dedicated mul_mat_vec + matmul via dequant_funcs

First Vulkan ternary type in ggml. Correctness: OM-125m TQ2_0 vs F16 top-12
logprobs identical to 4 decimals fully offloaded (float dequant path, no Q8_K
activation quant). Speed at 125m ~= F16 (overhead-bound at this scale); the
bandwidth win targets larger BitNet SKUs. MMQ/int-dot path intentionally not
wired yet.

Co-Authored-By: Claude Fable 5 <redacted>
* tests: enable TQ2_0 in backend-ops type lists

Vulkan now implements TQ2_0 (dequant, mul_mat_vec, mul_mm, get_rows); backends
without support skip via not-supported as usual. TQ1_0 stays disabled.

Co-Authored-By: Claude Fable 5 <redacted>
---------

Co-authored-by: Michael Trabalka <redacted>
Co-authored-by: Claude Fable 5 <redacted>
ggml/src/ggml-vulkan/ggml-vulkan.cpp
ggml/src/ggml-vulkan/vulkan-shaders/dequant_funcs.glsl
ggml/src/ggml-vulkan/vulkan-shaders/dequant_funcs_cm2.glsl
ggml/src/ggml-vulkan/vulkan-shaders/dequant_tq2_0.comp [new file with mode: 0644]
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_tq2_0.comp [new file with mode: 0644]
ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_funcs.glsl
ggml/src/ggml-vulkan/vulkan-shaders/types.glsl
ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp