Extract only the generation from calling make

[tapizquent]

When running a command like this:

arch -arm64 make -j && ./main -m ./models/open_llama_7b_700bt_ggml/ggml-model-q8_0.bin -p "A carpenter might cut some" -n 128

The output returns a bunch of information, that although great, might not be important for the end user.

ex:

I llama.cpp build info: 
I UNAME_S:  Darwin
I UNAME_P:  arm
I UNAME_M:  arm64
I CFLAGS:   -I.              -O3 -std=c11   -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -pthread -DGGML_USE_ACCELERATE
I CXXFLAGS: -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread
I LDFLAGS:   -framework Accelerate
I CC:       Apple clang version 14.0.3 (clang-1403.0.22.14.1)
I CXX:      Apple clang version 14.0.3 (clang-1403.0.22.14.1)

make: Nothing to be done for `default'.
main: build = 602 (3b126f6)
main: seed  = 1685400544
llama.cpp: loading model from ./models/open_llama_7b_700bt_ggml/ggml-model-q8_0.bin
llama_model_load_internal: format     = ggjt v3 (latest)
llama_model_load_internal: n_vocab    = 32000
llama_model_load_internal: n_ctx      = 512
llama_model_load_internal: n_embd     = 4096
llama_model_load_internal: n_mult     = 256
llama_model_load_internal: n_head     = 32
llama_model_load_internal: n_layer    = 32
llama_model_load_internal: n_rot      = 128
llama_model_load_internal: ftype      = 7 (mostly Q8_0)
llama_model_load_internal: n_ff       = 11008
llama_model_load_internal: n_parts    = 1
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size =    0.07 MB
llama_model_load_internal: mem required  = 8620.71 MB (+ 1026.00 MB per state)
.
llama_init_from_file: kv self size  =  256.00 MB

system_info: n_threads = 8 / 10 | AVX = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | VSX = 0 | 
sampling: repeat_last_n = 64, repeat_penalty = 1.100000, presence_penalty = 0.000000, frequency_penalty = 0.000000, top_k = 40, tfs_z = 1.000000, top_p = 0.950000, typical_p = 1.000000, temp = 0.800000, mirostat = 0, mirostat_lr = 0.100000, mirostat_ent = 5.000000
generate: n_ctx = 512, n_batch = 512, n_predict = 128, n_keep = 0


 A carpenter might cut a piece of timber 1 cm (0.4 in) across the grain and 30 cm (12 in) long, and another 30 cm (12 in) long, with the same width across the grain. The two pieces would be joined together by gluing, nailing or screwing them together.
When a piece of timber is cut across the grain, each side of the grain runs lengthwise, which reduces the amount of material to remove for the joint. When cutting across the grain, the board does not need to be sanded smooth, and does not need any glue or filler to create a joint
llama_print_timings:        load time =  7569.94 ms
llama_print_timings:      sample time =   111.03 ms /   128 runs   (    0.87 ms per token)
llama_print_timings: prompt eval time =  7510.38 ms /     9 tokens (  834.49 ms per token)
llama_print_timings:        eval time = 10765.59 ms /   127 runs   (   84.77 ms per token)
llama_print_timings:       total time = 18457.71 ms

How can I parse, split that out? Is there any way to do so with a flag?

[j-f1]

Except maybe for the make output, I believe everything besides the prompt/output text is sent to stderr, while the information you probably want is on stdout.