]> git.djapps.eu Git - pkg/ggml/sources/llama.cpp/commit
llama : add AWQ for llama, llama2, mpt, and mistral models (#4593)
authorNam D. Tran <redacted>
Wed, 27 Dec 2023 15:39:45 +0000 (22:39 +0700)
committerGitHub <redacted>
Wed, 27 Dec 2023 15:39:45 +0000 (17:39 +0200)
commitf6793491b5af6da75edad34d6f503ef86d31b09f
treeba50b7ae1aba91cb465a06970a11137baab7afcf
parent879b690a9e1eb1ab0a29b58236fc76978fb4d902
llama : add AWQ for llama, llama2, mpt, and mistral models (#4593)

* update: awq support llama-7b model

* update: change order

* update: benchmark results for llama2-7b

* update: mistral 7b v1 benchmark

* update: support 4 models

* fix: Readme

* update: ready for PR

* update: readme

* fix: readme

* update: change order import

* black

* format code

* update: work for bot mpt and awqmpt

* update: readme

* Rename to llm_build_ffn_mpt_awq

* Formatted other files

* Fixed params count

* fix: remove code

* update: more detail for mpt

* fix: readme

* fix: readme

* update: change folder architecture

* fix: common.cpp

* fix: readme

* fix: remove ggml_repeat

* update: cicd

* update: cicd

* uppdate: remove use_awq arg

* update: readme

* llama : adapt plamo to new ffn

ggml-ci

---------

Co-authored-by: Trần Đức Nam <redacted>
Co-authored-by: Le Hoang Anh <redacted>
Co-authored-by: Georgi Gerganov <redacted>
awq-py/README.md [new file with mode: 0644]
awq-py/awq/apply_awq.py [new file with mode: 0644]
awq-py/requirements.txt [new file with mode: 0644]
convert-hf-to-gguf.py
convert.py
gguf-py/gguf/constants.py
gguf-py/gguf/tensor_mapping.py
llama.cpp