[Pretrained transformers] small fix; only save once (#1817)
[Pretrained transformers] small fix; only save once (#1817)

* fix race condition

* no no reduction

facebookresearch/ParlAI,facebookresearch/ParlAI,facebookresearch/ParlAI,facebookresearch/ParlAI,facebookresearch/ParlAI