]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
llama : add early return for empty range (#8327)
authorDaniel Bevenius <redacted>
Sat, 6 Jul 2024 07:22:16 +0000 (09:22 +0200)
committerGitHub <redacted>
Sat, 6 Jul 2024 07:22:16 +0000 (10:22 +0300)
commit87e25a1d1bd26eb06d1cab9e2ee4e14a7a0be33c
treeedad2b97214648332bb9711bd20ad2fde4f6b2b1
parent213701b51a17175d0d326b566efc03f30ec7fbe6
llama : add early return for empty range (#8327)

* llama : add early return for empty range

This commit adds an early return to the llama_kv_cache_seq_add and
llama_kv_cache_seq_div functions.

The motivation for adding this is to avoid looping over the cache
when the range is empty. I ran into this when using the self-extend
feature in main.cpp.

Signed-off-by: Daniel Bevenius <redacted>
* llama : add static_cast to fix CI warning/error

This commit attempts to fix the following warning/error:

```console
src/llama.cpp:7271:31: error:
comparison of integer expressions of different signedness:
‘int’ and ‘uint32_t’ {aka ‘unsigned int’} [-Werror=sign-compare]
 7271 |                         if (i < hparams.n_layer_dense_lead) {
      |                             ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
This can be reproduced locally by setting -Wsign-compare in the
Makefile.

Signed-off-by: Daniel Bevenius <redacted>
* squash! llama : add early return for empty range

Remove the setting of cache.head to 0 when the range is empty.

Signed-off-by: Daniel Bevenius <redacted>
* Update src/llama.cpp

---------

Signed-off-by: Daniel Bevenius <redacted>
Co-authored-by: Georgi Gerganov <redacted>
src/llama.cpp