Avoid unnecessary DDP synchronization when gradient_accumulation_steps > 1 (#7742)
Avoid unnecessary DDP synchronization when gradient_accumulation_steps > 1 (#7742)

* use DDP no_sync when possible

* fix is_nlp_available addition mistake

* reformat trainer.py

* reformat trainer.py

* drop support for pytorch < 1.2

* return support for pytorch < 1.2
huggingface/transformers,huggingface/pytorch-transformers,huggingface/transformers,huggingface/transformers,huggingface/transformers