inference script throws error

[nashid]

During inference, we get the following error:

/XXXXXXXX/lib/python3.8/site-packages/numpy/core/_methods.py:190: RuntimeWarning: invalid value encountered in double_scalars
  ret = ret.dtype.type(ret / rcount)
Traceback (most recent call last):
  File "src/trainer/gpt_conut_trainer.py", line 263, in <module>
    generate_gpt_conut(vocab_file, trainer.model, input_file, identifier_txt_file, identifier_token_file, output_file, beam_size)
  File "/XXXXXXXX/cure/src/trainer/../tester/generator.py", line 89, in generate_gpt_conut
    generator.generate(output_file)
  File "/XXXXXXXX/src/trainer/../tester/generator.py", line 50, in generate
    for h in hypothesis:
UnboundLocalError: local variable 'hypothesis' referenced before assignment

Can you please share the working code for inference?

[jiang719]

I think this issue is caused by that lines 38-41 are not correctly executed. You may want to check if you load the model correctly, is the model an instance of GPTFConv or GPTCoNuT?

I added a try-catch to protect this, so the script won't crash.

[nashid]

This model is an instance of GPTCoNuT.