Reset loss to zero on logging in Trainer to avoid bfloat16 issues (#8561)
Reset loss to zero on logging in Trainer to avoid bfloat16 issues (#8561)

* make tr_loss regular float

* Revert "make tr_loss regular float"

This reverts commit c9d7ccfaf0c4387187b0841694f01ec0ffd5f4ba.

* reset loss at each logging step

* keep track of total loss with _total_loss_scalar

* add remaining tr_loss at the end
huggingface/transformers,huggingface/transformers,huggingface/transformers,huggingface/pytorch-transformers,huggingface/transformers