]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
spec: add EAGLE3 speculative decoding support (#18039)
authorRuixiang Wang <redacted>
Fri, 12 Jun 2026 07:21:06 +0000 (09:21 +0200)
committerGitHub <redacted>
Fri, 12 Jun 2026 07:21:06 +0000 (10:21 +0300)
commit88a39274ecf88ba11686acd357b59685b1cbf03d
tree350a48ee94f9fcab60faab12901bd57e79486604
parent85f99dca8b4539163e6858c9df7c9e8b0ad3632a
spec: add EAGLE3 speculative decoding support (#18039)

* llama : enable layer input extraction

* spec: support eagle3

* eagle3: fix params bug

* eagle3: support Gemma4 eagle3 from RedHatAI

* eagle3: set sync when get features from target

Co-authored-by: tnhnyzc <redacted>
* eagle3 : fix ubatch handling in embd_layer_inp extraction and encoder

Co-authored-by: Doğaç Eldenk <redacted>
* eagle3: adapt to upstream changes

* eagle3: fix rebase issues and adapt to upstream changes

* eagle3:exclude the eagle3 arch from test-llama-archs

* eagle3: fix editorconfig check failures

* eagle3: fix multi-seq issue in d2t vocab mapping

* cont : minor style / clean-up

* spec : remove `common_speculative_setup_draft_model()`

* llama : clean-up unused API

* eagle3: set d2t vocab mapping in decode graph

* cont : assert layer inputs are configured

* hparams : use n_embd_inp instead of n_embd_target_features

* eagle3: make output.weight optional and inherit from target model when needed

* haparams : generic norm-before-residual param

* llama-ext : consistent names

* cont : fix

* hparams : remove target_hidden_size

* cparams : rename output_layer_inp -> embeddings_layer_inp

* arch : reuse ATTN_NORM_2 instead of adding new hidden norm

* llama : clean-up names

* cont : add assert + comment

* Update conversion/llama.py

Co-authored-by: Sigbjørn Skjæret <redacted>
---------

Co-authored-by: Georgi Gerganov <redacted>
Co-authored-by: tnhnyzc <redacted>
Co-authored-by: Doğaç Eldenk <redacted>
Co-authored-by: Sigbjørn Skjæret <redacted>
27 files changed:
common/speculative.cpp
conversion/__init__.py
conversion/base.py
conversion/llama.py
convert_hf_to_gguf.py
gguf-py/gguf/constants.py
src/llama-arch.cpp
src/llama-arch.h
src/llama-context.cpp
src/llama-context.h
src/llama-cparams.h
src/llama-ext.h
src/llama-graph.cpp
src/llama-graph.h
src/llama-hparams.h
src/llama-model-loader.cpp
src/llama-model.cpp
src/llama-model.h
src/models/eagle3.cpp [new file with mode: 0644]
src/models/gemma4.cpp
src/models/llama.cpp
src/models/models.h
src/models/openai-moe.cpp
src/models/qwen3.cpp
src/models/qwen35.cpp
src/models/qwen3moe.cpp
tests/test-llama-archs.cpp