From: Josh Bleecher Snyder <redacted>
Date: Sat, 9 Mar 2024 16:50:03 +0000 (-0800)
Subject: whisper : make beam candidate sort more stable (whisper/1943)
X-Git-Tag: upstream/0.0.1642~870
X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=86961f54d35b39919780111dff69a1a040c81d0a;p=pkg%2Fggml%2Fsources%2Fggml

whisper : make beam candidate sort more stable (whisper/1943)

All else being otherwise equal, this encourages the beam candidate
selection to re-use the same decoder, which slightly
reduces the cache size.

I wouldn't expect it to make much of a performance difference,
but it helps when debug printing the cache and beam.

Added as part of understanding #1941.
---

diff --git a/examples/whisper/whisper.cpp b/examples/whisper/whisper.cpp
index 3459dd6e..84d292d3 100644
--- a/examples/whisper/whisper.cpp
+++ b/examples/whisper/whisper.cpp
@@ -5357,7 +5357,10 @@ int whisper_full_with_state(
                             beam_candidates.begin(),
                             beam_candidates.end(),
                             [](const beam_candidate & a, const beam_candidate & b) {
-                        return a.sequence.sum_logprobs_all > b.sequence.sum_logprobs_all;
+                        if (a.sequence.sum_logprobs_all != b.sequence.sum_logprobs_all) {
+                            return a.sequence.sum_logprobs_all > b.sequence.sum_logprobs_all;
+                        }
+                        return a.decoder_idx < b.decoder_idx;
                     });
 
                     uint32_t cur_c = 0;