Add check for num_shards when using distributed training
Add check for num_shards when using distributed training

Summary:
If num_shards = 1 and distributed training is on, then ring reduce fails when it looks for left pair to exchange information.
I also used the opportunity to do a small fix in my data loader benchmark

Differential Revision: D4513545

fbshipit-source-id: 7d3115b871a39b8ce7b55553394b607d16e08b74

sf-wind/caffe2,pietern/caffe2,xzturn/caffe2,bwasti/caffe2,sf-wind/caffe2,Yangqing/caffe2,xzturn/caffe2,davinwang/caffe2,xzturn/caffe2,pietern/caffe2,bwasti/caffe2,davinwang/caffe2,Yangqing/caffe2,Yangqing/caffe2,bwasti/caffe2,xzturn/caffe2,bwasti/caffe2,xzturn/caffe2,davinwang/caffe2,bwasti/caffe2,Yangqing/caffe2,davinwang/caffe2,pietern/caffe2,caffe2/caffe2,Yangqing/caffe2,pietern/caffe2,sf-wind/caffe2,sf-wind/caffe2,davinwang/caffe2,sf-wind/caffe2,pietern/caffe2