Debugging and removing redundant lines
#8
by
unuu
- opened
I think these changes are needed and fixes a bug but I don't know what testing process was done.
Please see this issue and if deemed valid, I will move the explanations here.
Thanks for your advice! You can pass 'use_cache=False' or downgrade the transformers library to version 4.53.0 to avoid this bug. In our model, we use our own cache but not hf's cache.
generated_ids = model.generate(
inputs=inputs['input_ids'].to(device),
**inputs,
max_new_tokens=128,
use_cache=False
)