You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
llama.cpp often continues to generate tokens even I specify the necessary reverse prompts: '### Human:' or 'Character:'
I am running in interactive mode, sometimes I get back the control, but often it takes one or more turns where llama.cpp skips to detect that the reverse prompt strings are in the token stream and hand back control.
Impact: bad chat experience.
Environment: MSYS2/Mingw64. Latest llama.cpp version from master.
Interestingly I don't experience the same problem if I compile with VS2019.
Model used: Vicuna 13B:
ggml-vicuna-13B-1.1-q4_0.bin
Tried specifying the reverse prompt both with single and double quotes - same result.
The text was updated successfully, but these errors were encountered:
1OOyan
changed the title
Text generation continues even when reverse prompts are specified.
MSYS2/MinGW64 Text generation continues even when reverse prompts are specified.
Apr 13, 2023
I ran into the same problem, but using '###' for the reverse prompt stopped the output correctly. However, '### ' (adding a space at the end) did not stop it, so I think there is a high possibility that there is a problem with handling spaces.
llama.cpp often continues to generate tokens even I specify the necessary reverse prompts: '### Human:' or 'Character:'
I am running in interactive mode, sometimes I get back the control, but often it takes one or more turns where llama.cpp skips to detect that the reverse prompt strings are in the token stream and hand back control.
Impact: bad chat experience.
Environment: MSYS2/Mingw64. Latest llama.cpp version from master.
Interestingly I don't experience the same problem if I compile with VS2019.
Model used: Vicuna 13B:
ggml-vicuna-13B-1.1-q4_0.bin
Command line:
./main.exe -i -f prompts/chat-with-samantha.txt -m ggml-vicuna-13B-1.1-q4_0.bin -r 'You:' -r '### Human:' -r 'Stan:' --color -t 8 --keep -1 -c 2048 -n -1
Tried specifying the reverse prompt both with single and double quotes - same result.
The text was updated successfully, but these errors were encountered: