]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
Stream save llama context data to file instead of allocating entire buffer upfront...
authorl3utterfly <redacted>
Fri, 4 Aug 2023 11:29:52 +0000 (19:29 +0800)
committerGitHub <redacted>
Fri, 4 Aug 2023 11:29:52 +0000 (13:29 +0200)
commit415e99fec27be5a2e4283f1937afd17eb33fbd66
treeb16b4735a94fa5a5fc76cb4fef691fbe6c210489
parentff966e7ca6af127c9405523cdb07ef8fa01bf6d6
Stream save llama context data to file instead of allocating entire buffer upfront (#2488)

* added stream saving context data to file to avoid allocating unnecessary amounts of memory

* generalised copying state data to file or buffer

* added comments explaining how copy_state_data works

* fixed trailing whitespaces

* fixed save load state example

* updated save load state to use public function in llama.cpp

* - restored breakage of the llama_copy_state_data API
- moved new logic for copying llama state data to internal function

* fixed function declaration order

* restored save load state example

* fixed whitepace

* removed unused llama-util.h include

* Apply suggestions from code review

Co-authored-by: slaren <redacted>
* Apply code review suggestions

Co-authored-by: slaren <redacted>
---------

Co-authored-by: slaren <redacted>
llama-util.h
llama.cpp