]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
mtmd : add ultravox audio input (#13623)
authorXuan-Son Nguyen <redacted>
Thu, 22 May 2025 18:42:48 +0000 (20:42 +0200)
committerGitHub <redacted>
Thu, 22 May 2025 18:42:48 +0000 (20:42 +0200)
commit797990c4bca0dca5be295c63e3fb2800dc0a69c2
tree5c0fbe3e9b2c7b4e601d4b545e87c50871eb6667
parentab86335760ebb441574eb47f886fa1ee302e2131
mtmd : add ultravox audio input (#13623)

* convert ok, load ok

* warmup ok

* test

* still does not work?

* fix padding

* temporary give up

* fix merge conflict

* build_ultravox()

* rm test

* fix merge conflict

* add necessary mtmd APIs

* first working version (only 4s of audio)

* will this monster compile?

* fix compile

* please compile

* fPIC

* fix windows

* various fixes

* clean up audio_helpers

* fix conversion

* add some debug stuff

* long audio input ok

* adapt the api

* add --audio arg

* final touch UX

* add miniaudio to readme

* fix typo

* refactor kv metadata

* mtmd_default_marker()
21 files changed:
.editorconfig
README.md
common/arg.cpp
common/common.h
convert_hf_to_gguf.py
docs/multimodal.md
gguf-py/gguf/constants.py
gguf-py/gguf/gguf_writer.py
gguf-py/gguf/tensor_mapping.py
tools/mtmd/CMakeLists.txt
tools/mtmd/clip-impl.h
tools/mtmd/clip.cpp
tools/mtmd/clip.h
tools/mtmd/miniaudio.h [new file with mode: 0644]
tools/mtmd/mtmd-audio.cpp [new file with mode: 0644]
tools/mtmd/mtmd-audio.h [new file with mode: 0644]
tools/mtmd/mtmd-cli.cpp
tools/mtmd/mtmd-helper.cpp
tools/mtmd/mtmd.cpp
tools/mtmd/mtmd.h
tools/server/utils.hpp