Failed to load model

[coogle]

Hello,

I was playing with this trying to get it to work, but couldn't get the model to load. I used these instructions on my MBP M1 for the 13B model:

https://til.simonwillison.net/llms/llama-7b-m2

I get a "unknown tensor" error as shown:

./main \
  -m ./models/13B/ggml-model-q4_0.bin \
  -t 8 \
  -n 128 \
  -p 'The first person to go to space was '
main: seed = 1678602312
llama_model_load: loading model from './models/13B/ggml-model-q4_0.bin' - please wait ...
llama_model_load: n_vocab = 32000
llama_model_load: n_ctx   = 512
llama_model_load: n_embd  = 5120
llama_model_load: n_mult  = 256
llama_model_load: n_head  = 40
llama_model_load: n_layer = 40
llama_model_load: n_rot   = 128
llama_model_load: f16     = 2
llama_model_load: n_ff    = 13824
llama_model_load: n_parts = 2
llama_model_load: ggml ctx size = 8559.49 MB
llama_model_load: memory_size =   800.00 MB, n_mem = 20480
llama_model_load: loading model part 1/2 from './models/13B/ggml-model-q4_0.bin'
llama_model_load: ............................................. done
llama_model_load: model size =  3880.49 MB / num tensors = 363
llama_model_load: loading model part 2/2 from './models/13B/ggml-model-q4_0.bin.1'
llama_model_load: unknown tensor '' in model file
main: failed to load model from './models/13B/ggml-model-q4_0.bin'
llama_model_load: %               

Any suggestions would be great! Thanks for working on this I'm excited to get it running.

[ggerganov]

You haven't quantized the ./models/13B/ggml-model-q4_0.bin.1 part of the model.
See the Readme - larger models produce several parts, each stored in a separate file with an integer suffix. At the moment, you need to quantize these parts individually

[coogle]

Yep that was it, thanks!