From: Georgi Gerganov Date: Sat, 25 Mar 2023 18:36:52 +0000 (+0200) Subject: Move chat scripts into "./examples" X-Git-Tag: gguf-v0.4.0~1103 X-Git-Url: https://git.djapps.eu/?a=commitdiff_plain;h=55ad42af845127bd0eb0c1f36f327ecec83f4bca;p=pkg%2Fggml%2Fsources%2Fllama.cpp Move chat scripts into "./examples" --- diff --git a/README.md b/README.md index 8a84324b..9ba6241d 100644 --- a/README.md +++ b/README.md @@ -179,7 +179,10 @@ Here is an example few-shot interaction, invoked with the command ```bash # default arguments using 7B model -./chat.sh +./examples/chat.sh + +# advanced chat with 13B model +./examples/chat-13B.sh # custom arguments using 13B model ./main -m ./models/13B/ggml-model-q4_0.bin -n 256 --repeat_penalty 1.0 --color -i -r "User:" -f prompts/chat-with-bob.txt @@ -195,7 +198,7 @@ Note the use of `--color` to distinguish between user input and generated text. 2. Run the `main` tool like this: ``` -./main -m ./models/ggml-alpaca-7b-q4.bin --color -f ./prompts/alpaca.txt -ins +./examples/alpaca.sh ``` Sample run: diff --git a/alpaca.sh b/alpaca.sh deleted file mode 100755 index d8a9f456..00000000 --- a/alpaca.sh +++ /dev/null @@ -1,6 +0,0 @@ -#!/bin/bash -# -# Temporary script - will be removed in the future -# - -./main -m ./models/ggml-alpaca-7b-q4.bin --color -f ./prompts/alpaca.txt -ins -b 256 --top_k 10000 --temp 0.2 --repeat_penalty 1 -t 7 diff --git a/chat.sh b/chat.sh deleted file mode 100755 index 5531315b..00000000 --- a/chat.sh +++ /dev/null @@ -1,6 +0,0 @@ -#!/bin/bash -# -# Temporary script - will be removed in the future -# - -./main -m ./models/7B/ggml-model-q4_0.bin -b 128 -n 256 --repeat_penalty 1.0 --color -i -r "User:" -f prompts/chat-with-bob.txt diff --git a/examples/alpaca.sh b/examples/alpaca.sh new file mode 100755 index 00000000..4c9aa507 --- /dev/null +++ b/examples/alpaca.sh @@ -0,0 +1,10 @@ +#!/bin/bash + +# +# Temporary script - will be removed in the future +# + +cd `dirname $0` +cd .. + +./main -m ./models/ggml-alpaca-7b-q4.bin --color -f ./prompts/alpaca.txt -ins -b 256 --top_k 10000 --temp 0.2 --repeat_penalty 1 -t 7 diff --git a/examples/chat-13B.sh b/examples/chat-13B.sh new file mode 100755 index 00000000..4265d7b6 --- /dev/null +++ b/examples/chat-13B.sh @@ -0,0 +1,53 @@ +#!/bin/bash + +cd "$(dirname "$0")/.." || exit + +MODEL="${MODEL:-./models/13B/ggml-model-q4_0.bin}" +USER_NAME="${USER_NAME:-User}" +AI_NAME="${AI_NAME:-ChatLLaMa}" + +# Adjust to the number of CPU cores you want to use. +N_THREAD="${N_THREAD:-8}" +# Number of tokens to predict (made it larger than default because we want a long interaction) +N_PREDICTS="${N_PREDICTS:-2048}" + +# Note: you can also override the generation options by specifying them on the command line: +# For example, override the context size by doing: ./chatLLaMa --ctx_size 1024 +GEN_OPTIONS="${GEN_OPTIONS:---ctx_size 2048 --temp 0.7 --top_k 40 --top_p 0.5 --repeat_last_n 256 --batch_size 1024 --repeat_penalty 1.17647}" + +# shellcheck disable=SC2086 # Intended splitting of GEN_OPTIONS +./main $GEN_OPTIONS \ + --model "$MODEL" \ + --threads "$N_THREAD" \ + --n_predict "$N_PREDICTS" \ + --color --interactive \ + --reverse-prompt "${USER_NAME}:" \ + --prompt " +Text transcript of a never ending dialog, where ${USER_NAME} interacts with an AI assistant named ${AI_NAME}. +${AI_NAME} is helpful, kind, honest, friendly, good at writing and never fails to answer ${USER_NAME}’s requests immediately and with details and precision. +There are no annotations like (30 seconds passed...) or (to himself), just what ${USER_NAME} and ${AI_NAME} say aloud to each other. +The dialog lasts for years, the entirety of it is shared below. It's 10000 pages long. +The transcript only includes text, it does not include markup like HTML and Markdown. + +$USER_NAME: Hello, $AI_NAME! +$AI_NAME: Hello $USER_NAME! How may I help you today? +$USER_NAME: What time is it? +$AI_NAME: It is $(date +%H:%M). +$USER_NAME: What year is it? +$AI_NAME: We are in $(date +%Y). +$USER_NAME: Please tell me the largest city in Europe. +$AI_NAME: The largest city in Europe is Moscow, the capital of Russia. +$USER_NAME: What can you tell me about Moscow? +$AI_NAME: Moscow, on the Moskva River in western Russia, is the nation’s cosmopolitan capital. In its historic core is the Kremlin, a complex that’s home to the president and tsarist treasures in the Armoury. Outside its walls is Red Square, Russia’s symbolic center. +$USER_NAME: What is a cat? +$AI_NAME: A cat is a domestic species of small carnivorous mammal. It is the only domesticated species in the family Felidae. +$USER_NAME: How do I pass command line arguments to a Node.js program? +$AI_NAME: The arguments are stored in process.argv. + + argv[0] is the path to the Node. js executable. + argv[1] is the path to the script file. + argv[2] is the first argument passed to the script. + argv[3] is the second argument passed to the script and so on. +$USER_NAME: Name a color. +$AI_NAME: Blue +$USER_NAME:" "$@" diff --git a/examples/chat.sh b/examples/chat.sh new file mode 100755 index 00000000..2b6a63e3 --- /dev/null +++ b/examples/chat.sh @@ -0,0 +1,10 @@ +#!/bin/bash + +# +# Temporary script - will be removed in the future +# + +cd `dirname $0` +cd .. + +./main -m ./models/7B/ggml-model-q4_0.bin -b 128 -n 256 --repeat_penalty 1.0 --color -i -r "User:" -f prompts/chat-with-bob.txt diff --git a/examples/chatLLaMa b/examples/chatLLaMa deleted file mode 100755 index 4265d7b6..00000000 --- a/examples/chatLLaMa +++ /dev/null @@ -1,53 +0,0 @@ -#!/bin/bash - -cd "$(dirname "$0")/.." || exit - -MODEL="${MODEL:-./models/13B/ggml-model-q4_0.bin}" -USER_NAME="${USER_NAME:-User}" -AI_NAME="${AI_NAME:-ChatLLaMa}" - -# Adjust to the number of CPU cores you want to use. -N_THREAD="${N_THREAD:-8}" -# Number of tokens to predict (made it larger than default because we want a long interaction) -N_PREDICTS="${N_PREDICTS:-2048}" - -# Note: you can also override the generation options by specifying them on the command line: -# For example, override the context size by doing: ./chatLLaMa --ctx_size 1024 -GEN_OPTIONS="${GEN_OPTIONS:---ctx_size 2048 --temp 0.7 --top_k 40 --top_p 0.5 --repeat_last_n 256 --batch_size 1024 --repeat_penalty 1.17647}" - -# shellcheck disable=SC2086 # Intended splitting of GEN_OPTIONS -./main $GEN_OPTIONS \ - --model "$MODEL" \ - --threads "$N_THREAD" \ - --n_predict "$N_PREDICTS" \ - --color --interactive \ - --reverse-prompt "${USER_NAME}:" \ - --prompt " -Text transcript of a never ending dialog, where ${USER_NAME} interacts with an AI assistant named ${AI_NAME}. -${AI_NAME} is helpful, kind, honest, friendly, good at writing and never fails to answer ${USER_NAME}’s requests immediately and with details and precision. -There are no annotations like (30 seconds passed...) or (to himself), just what ${USER_NAME} and ${AI_NAME} say aloud to each other. -The dialog lasts for years, the entirety of it is shared below. It's 10000 pages long. -The transcript only includes text, it does not include markup like HTML and Markdown. - -$USER_NAME: Hello, $AI_NAME! -$AI_NAME: Hello $USER_NAME! How may I help you today? -$USER_NAME: What time is it? -$AI_NAME: It is $(date +%H:%M). -$USER_NAME: What year is it? -$AI_NAME: We are in $(date +%Y). -$USER_NAME: Please tell me the largest city in Europe. -$AI_NAME: The largest city in Europe is Moscow, the capital of Russia. -$USER_NAME: What can you tell me about Moscow? -$AI_NAME: Moscow, on the Moskva River in western Russia, is the nation’s cosmopolitan capital. In its historic core is the Kremlin, a complex that’s home to the president and tsarist treasures in the Armoury. Outside its walls is Red Square, Russia’s symbolic center. -$USER_NAME: What is a cat? -$AI_NAME: A cat is a domestic species of small carnivorous mammal. It is the only domesticated species in the family Felidae. -$USER_NAME: How do I pass command line arguments to a Node.js program? -$AI_NAME: The arguments are stored in process.argv. - - argv[0] is the path to the Node. js executable. - argv[1] is the path to the script file. - argv[2] is the first argument passed to the script. - argv[3] is the second argument passed to the script and so on. -$USER_NAME: Name a color. -$AI_NAME: Blue -$USER_NAME:" "$@"