]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
model: add Hy3 (hy_v3) support with MTP speculative decoding (#25395)
authorSatinder Grewal <redacted>
Mon, 13 Jul 2026 22:31:04 +0000 (10:31 +1200)
committerGitHub <redacted>
Mon, 13 Jul 2026 22:31:04 +0000 (00:31 +0200)
commit2969d6d15d67a08e7b83f26164b15350c79c5248
tree5730a66cf5c8739a73f0ee9ae972db19350a5b5d
parent6eddde06a4f25d55d538b5d15628dcc2b6882147
model: add Hy3 (hy_v3) support with MTP speculative decoding (#25395)

* model: add Hy3 (hy_v3) architecture support

Adds Tencent Hunyuan 3 (HF architecture HYV3ForCausalLM, GGUF arch
hy_v3): a MoE decoder stack with per-head Q/K RMSNorm, a sigmoid
router with expert selection bias, an always-active ungated shared
expert, and leading dense block(s) (first_k_dense_replace).

The base implementation is ported from charlie12345's fork
(https://github.com/charlie12345/ROCmFPX, src/models/hyv3.cpp),
adapted to current mainline APIs (hparams.n_layer(), build_qkv,
build_moe_ffn with fused gate_up + scale tensors, output_s).

Note: blk.N.exp_probs_b is stored without a .bias suffix for
compatibility with existing hy_v3 GGUFs produced by that fork.

Co-Authored-By: charlie12345 <redacted>
Co-authored-by: Piotr Wilkin <redacted>
Assisted-by: Claude Fable 5
16 files changed:
common/chat-auto-parser-generator.cpp
common/chat-auto-parser.h
common/chat-diff-analyzer.cpp
common/jinja/value.cpp
conversion/__init__.py
conversion/hunyuan.py
gguf-py/gguf/constants.py
models/templates/tencent-Hy3.jinja [new file with mode: 0644]
src/llama-arch.cpp
src/llama-arch.h
src/llama-model.cpp
src/models/hy-v3.cpp [new file with mode: 0644]
src/models/models.h
tests/test-chat-auto-parser.cpp
tests/test-jinja.cpp
tests/test-llama-archs.cpp