]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
model : add HunyuanOCR support (#21395)
authorRichard Davison <redacted>
Sun, 5 Apr 2026 21:32:14 +0000 (23:32 +0200)
committerGitHub <redacted>
Sun, 5 Apr 2026 21:32:14 +0000 (23:32 +0200)
commitaf76639f728c69c74c873cc45f038aaa17afd09e
tree0ee1e644f6c3bf95a1e76371bf949cdafb75597c
parent761797ffdf2ce3f118e82c663b1ad7d935fbd656
model : add HunyuanOCR support (#21395)

* HunyuanOCR: add support for text and vision models

- Add HunyuanOCR vision projector (perceiver-based) with Conv2d merge
- Add separate HUNYUAN_OCR chat template (content-before-role format)
- Handle HunyuanOCR's invalid pad_token_id=-1 in converter
- Fix EOS/EOT token IDs from generation_config.json
- Support xdrope RoPE scaling type
- Add tensor mappings for perceiver projector (mm.before_rms, mm.after_rms, etc.)
- Register HunYuanVLForConditionalGeneration for both text and mmproj conversion

* fix proper mapping

* Update gguf-py/gguf/tensor_mapping.py

Co-authored-by: Xuan-Son Nguyen <redacted>
* Update tools/mtmd/clip.cpp

Co-authored-by: Xuan-Son Nguyen <redacted>
* address comments

* update

* Fix typecheck

* Update convert_hf_to_gguf.py

Co-authored-by: Sigbjørn Skjæret <redacted>
* Update convert_hf_to_gguf.py

Co-authored-by: Sigbjørn Skjæret <redacted>
* Update convert_hf_to_gguf.py

Co-authored-by: Sigbjørn Skjæret <redacted>
* Update convert_hf_to_gguf.py

Co-authored-by: Sigbjørn Skjæret <redacted>
---------

Co-authored-by: Xuan-Son Nguyen <redacted>
Co-authored-by: Sigbjørn Skjæret <redacted>
12 files changed:
convert_hf_to_gguf.py
gguf-py/gguf/constants.py
gguf-py/gguf/tensor_mapping.py
src/llama-chat.cpp
src/llama-chat.h
tools/mtmd/CMakeLists.txt
tools/mtmd/clip-impl.h
tools/mtmd/clip-model.h
tools/mtmd/clip.cpp
tools/mtmd/models/hunyuanocr.cpp [new file with mode: 0644]
tools/mtmd/models/models.h
tools/mtmd/mtmd.cpp