]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
vulkan: add TQ2_0 (ternary) support (#25850)
authormichaeltrabalka-tech <redacted>
Wed, 12 Aug 2026 05:07:23 +0000 (01:07 -0400)
committerGitHub <redacted>
Wed, 12 Aug 2026 05:07:23 +0000 (08:07 +0300)
commita7cd2f0e98ffa4e939276287be1fa76891908d2e
tree87dbb82d4c4e24073bd10dfe200f6dd2c9f0dea4
parent55f453b92452c9a103ae963543addce02e800f90
vulkan: add TQ2_0 (ternary) support (#25850)

* vulkan: TQ2_0 (ternary) support — dequant + dedicated mul_mat_vec + matmul via dequant_funcs

First Vulkan ternary type in ggml. Correctness: OM-125m TQ2_0 vs F16 top-12
logprobs identical to 4 decimals fully offloaded (float dequant path, no Q8_K
activation quant). Speed at 125m ~= F16 (overhead-bound at this scale); the
bandwidth win targets larger BitNet SKUs. MMQ/int-dot path intentionally not
wired yet.

Co-Authored-By: Claude Fable 5 <redacted>
* tests: enable TQ2_0 in backend-ops type lists

Vulkan now implements TQ2_0 (dequant, mul_mat_vec, mul_mm, get_rows); backends
without support skip via not-supported as usual. TQ1_0 stays disabled.

Co-Authored-By: Claude Fable 5 <redacted>
---------

Co-authored-by: Michael Trabalka <redacted>
Co-authored-by: Claude Fable 5 <redacted>
ggml/src/ggml-vulkan/ggml-vulkan.cpp
ggml/src/ggml-vulkan/vulkan-shaders/dequant_funcs.glsl
ggml/src/ggml-vulkan/vulkan-shaders/dequant_funcs_cm2.glsl
ggml/src/ggml-vulkan/vulkan-shaders/dequant_tq2_0.comp [new file with mode: 0644]
ggml/src/ggml-vulkan/vulkan-shaders/mul_mat_vec_tq2_0.comp [new file with mode: 0644]
ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_funcs.glsl
ggml/src/ggml-vulkan/vulkan-shaders/types.glsl
ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp
tests/test-backend-ops.cpp