git.djapps.eu Git - pkg/ggml/sources/ggml/commit

author	Lukas Möller <redacted>
	Wed, 17 May 2023 19:58:21 +0000 (21:58 +0200)
committer	GitHub <redacted>
	Wed, 17 May 2023 19:58:21 +0000 (22:58 +0300)
commit	1d6a133b098e3b038a7e2c544941b5288ca76bc8
tree	3816f06744b2225d8cc83033253058db01221820	tree
parent	74705055853f7922e9622bdd0a1ebde2b8f57431	commit \| diff

examples : sample replit + MPT inference (#145)

* Add replit model

* Add unigram tokenization support

* Remove debug log

* Port alibi attn bias fix

* Remove torch input

* Fix hardcoded path

* Remove unsupported hyperparams

* Add mpt

* Add replit quantization script

* Remove debug print

* Add quantization support to mpt

* Reformat

* Remove trailing return type

* Implement stylistic changes

* use f16 in k/v memory calculations for replit/mpt

* Update context size calculation

* Add clip_qkv and alibi_bias_max support

* fix clamping implementation, remove implicit conversions

* Fix qkv if condition

* Fix replit context size calculation

* Potentially fix gcc compilation error

* Fix warning

* Adjust object overhead

* Remove dead code

examples/CMakeLists.txt		diff \| blob \| history
examples/mpt/CMakeLists.txt	[new file with mode: 0644]	blob
examples/mpt/convert-h5-to-ggml.py	[new file with mode: 0644]	blob
examples/mpt/main.cpp	[new file with mode: 0644]	blob
examples/mpt/quantize.cpp	[new file with mode: 0644]	blob
examples/replit/CMakeLists.txt	[new file with mode: 0644]	blob
examples/replit/convert-h5-to-ggml.py	[new file with mode: 0644]	blob
examples/replit/main.cpp	[new file with mode: 0644]	blob
examples/replit/quantize.cpp	[new file with mode: 0644]	blob
include/ggml/ggml.h		diff \| blob \| history
src/ggml.c		diff \| blob \| history