]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
mtmd, llama : Update HunyuanVL vision-language model support (#22037)
authormanayang <redacted>
Wed, 22 Apr 2026 09:58:43 +0000 (17:58 +0800)
committerGitHub <redacted>
Wed, 22 Apr 2026 09:58:43 +0000 (11:58 +0200)
commit7bfe60fdf929ae569b81bbbce7ff7be5a1f8e354
tree4d80dbdb46312223a6ba9ec63b1e8781a29bdeb0
parent750579ff14198fe964ab7fc5565b1d77600deab4
mtmd, llama : Update HunyuanVL vision-language model support (#22037)

* mtmd, llama : add HunyuanVL vision-language model support

- add LLM_ARCH_HUNYUAN_VL with M-RoPE (XD-RoPE) support
- add PROJECTOR_TYPE_HUNYUANVL with PatchMerger vision encoder
- add HunyuanVL-specific M-RoPE position encoding for image tokens
- add GGUF conversion for HunyuanVL vision and text models
- add smoke test in tools/mtmd/tests.sh

* fix: fix HunyuanVL XD-RoPE h/w section order

* fix: Remove redundant code

* convert : fix HunyuanOCR / HunyuanVL conversion
 - Tested locally: both HunyuanOCR and HunyuanVL-4B convert to GGUF
 - successfully and produce correct inference output on Metal (F16 / Q8_0).

* clip : fix -Werror=misleading-indentation in bilinear resize

* fix CI: convert_hf_to_gguf type check error
 - convert_hf_to_gguf.py: give HunyuanVLTextModel.__init__ an explicit `dir_model: Path` parameter so ty can infer the type for load_hparams instead of reporting `Unknown | None`.

---------

Co-authored-by: wendadawen <redacted>
13 files changed:
convert_hf_to_gguf.py
gguf-py/gguf/constants.py
gguf-py/gguf/gguf_writer.py
src/llama-arch.cpp
src/llama-arch.h
src/llama-hparams.h
src/llama-model.cpp
src/models/hunyuan-dense.cpp
tools/mtmd/clip-impl.h
tools/mtmd/clip.cpp
tools/mtmd/models/hunyuanocr.cpp
tools/mtmd/mtmd.cpp
tools/mtmd/tests.sh