]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
model-conversion : add support for SentenceTransformers (#16387)
authorDaniel Bevenius <redacted>
Thu, 9 Oct 2025 12:35:22 +0000 (14:35 +0200)
committerGitHub <redacted>
Thu, 9 Oct 2025 12:35:22 +0000 (14:35 +0200)
commit56b4795842d852152222ca7a2d4304008facf1b9
treef14678f0d83eecc000739b8f54d1732059b3d84a
parent2c0d875ae6c6043fbbafd2831e160955e9ca6af1
model-conversion : add support for SentenceTransformers (#16387)

* model-conversion : add support for SentenceTransformers

This commit adds support for models that use SentenceTransformer layers.

The motivation for this is that if converted model includes any of the
numbered layers specified in the original models repository then these
changes enable these models to be used and verified. Currently the
model-conversion only support the base model output without any of
the additional transformation layers.

Usage:
Convert the model that also includes the SentenceTransformer layers:
```console
(venv) $ export EMBEDDING_MODEL_PATH="~/google/embeddinggemma-300M"
(venv) make embedding-convert-model
```

Verify the produced embeddings from the converted model against the
original model embeddings:
```console
(venv) make embedding-verify-logits-st
```

The original model can be run using SentenceTransformer:
```console
(venv) make embedding-run-original-model-st
```

Run the converted model using "SentenceTransformer" layers whic
enables pooling and normalization:
```console
(venv) make embedding-run-converted-model-st
```

* add model-conversion example requirements

* add support for -st flag in embedding model conversion

This commit add support for the -st flag in the embedding model
conversion script. This will enable models to be converted using
sentence transformers dense layers.
examples/model-conversion/Makefile
examples/model-conversion/README.md
examples/model-conversion/logits.cpp
examples/model-conversion/requirements.txt
examples/model-conversion/scripts/embedding/convert-model.sh
examples/model-conversion/scripts/embedding/run-converted-model.sh
examples/model-conversion/scripts/embedding/run-original-model.py
examples/model-conversion/scripts/utils/semantic_check.py
requirements/requirements-all.txt