Reuse common world for allreduce/broadcast
Reuse common world for allreduce/broadcast

Summary:
The common worlds can be reused without performance impact as long as
there is a guarantee that no two algorithm instances are using it at
any given time. Since we know the ordering and the maximum
parallelism, we can cycle through common worlds, and reuse them
accordingly.

Differential Revision: D4896779

fbshipit-source-id: 164e1727692eab904fa6879a9f91a3e8332a2e30

Yangqing/caffe2,pietern/caffe2,davinwang/caffe2,bwasti/caffe2,sf-wind/caffe2,Yangqing/caffe2,bwasti/caffe2,Yangqing/caffe2,xzturn/caffe2,Yangqing/caffe2,sf-wind/caffe2,davinwang/caffe2,pietern/caffe2,sf-wind/caffe2,xzturn/caffe2,bwasti/caffe2,bwasti/caffe2,davinwang/caffe2,Yangqing/caffe2,davinwang/caffe2,davinwang/caffe2,xzturn/caffe2,pietern/caffe2,sf-wind/caffe2,xzturn/caffe2,caffe2/caffe2,pietern/caffe2,sf-wind/caffe2,bwasti/caffe2,xzturn/caffe2,pietern/caffe2