Reuse GPUs for multiple ranks in NcclManager test.
Reuse GPUs for multiple ranks in NcclManager test.

Since NCCL 2.1.2, NCCL uses cooperative group launch by default which
results in deadlocks in the NcclManager test if number of ranks > number of
GPUs.  Setting NCCL_LAUNCH_MODE to PARALLEL, which was the default before NCCL
2.1.2, fixes this.

PiperOrigin-RevId: 224531851

ghchinoy/tensorflow,paolodedios/tensorflow,yongtang/tensorflow,karllessard/tensorflow,apark263/tensorflow,xzturn/tensorflow,xzturn/tensorflow,theflofly/tensorflow,jhseu/tensorflow,renyi533/tensorflow,freedomtan/tensorflow,annarev/tensorflow,paolodedios/tensorflow,jhseu/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,paolodedios/tensorflow,alsrgv/tensorflow,gautam1858/tensorflow,yongtang/tensorflow,jhseu/tensorflow,davidzchen/tensorflow,petewarden/tensorflow,arborh/tensorflow,jhseu/tensorflow,gunan/tensorflow,aam-at/tensorflow,annarev/tensorflow,cxxgtxy/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,ghchinoy/tensorflow,davidzchen/tensorflow,tensorflow/tensorflow-pywrap_saved_model,chemelnucfin/tensorflow,yongtang/tensorflow,renyi533/tensorflow,hfp/tensorflow-xsmm,alsrgv/tensorflow,renyi533/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,karllessard/tensorflow,Intel-tensorflow/tensorflow,chemelnucfin/tensorflow,gautam1858/tensorflow,DavidNorman/tensorflow,karllessard/tensorflow,Intel-tensorflow/tensorflow,hfp/tensorflow-xsmm,jbedorf/tensorflow,yongtang/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,asimshankar/tensorflow,arborh/tensorflow,karllessard/tensorflow,ppwwyyxx/tensorflow,ageron/tensorflow,ageron/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,theflofly/tensorflow,jendap/tensorflow,tensorflow/tensorflow-pywrap_saved_model,ghchinoy/tensorflow,yongtang/tensorflow,asimshankar/tensorflow,tensorflow/tensorflow,paolodedios/tensorflow,freedomtan/tensorflow,asimshankar/tensorflow,annarev/tensorflow,ageron/tensorflow,renyi533/tensorflow,gunan/tensorflow,gunan/tensorflow,ppwwyyxx/tensorflow,freedomtan/tensorflow,hfp/tensorflow-xsmm,renyi533/tensorflow,DavidNorman/tensorflow,petewarden/tensorflow,jendap/tensorflow,jendap/tensorflow,hfp/tensorflow-xsmm,jbedorf/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,kevin-coder/tensorflow-fork,aldian/tensorflow,alsrgv/tensorflow,chemelnucfin/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,tensorflow/tensorflow,frreiss/tensorflow-fred,theflofly/tensorflow,jhseu/tensorflow,cxxgtxy/tensorflow,paolodedios/tensorflow,tensorflow/tensorflow,xzturn/tensorflow,apark263/tensorflow,frreiss/tensorflow-fred,arborh/tensorflow,kevin-coder/tensorflow-fork,arborh/tensorflow,Intel-tensorflow/tensorflow,jhseu/tensorflow,theflofly/tensorflow,gautam1858/tensorflow,Bismarrck/tensorflow,frreiss/tensorflow-fred,freedomtan/tensorflow,apark263/tensorflow,arborh/tensorflow,renyi533/tensorflow,jendap/tensorflow,apark263/tensorflow,jendap/tensorflow,xzturn/tensorflow,ppwwyyxx/tensorflow,renyi533/tensorflow,kevin-coder/tensorflow-fork,paolodedios/tensorflow,chemelnucfin/tensorflow,frreiss/tensorflow-fred,DavidNorman/tensorflow,Bismarrck/tensorflow,xzturn/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,sarvex/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,chemelnucfin/tensorflow,sarvex/tensorflow,chemelnucfin/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,adit-chandra/tensorflow,tensorflow/tensorflow,ppwwyyxx/tensorflow,aam-at/tensorflow,ghchinoy/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,freedomtan/tensorflow,apark263/tensorflow,ghchinoy/tensorflow,cxxgtxy/tensorflow,renyi533/tensorflow,theflofly/tensorflow,kevin-coder/tensorflow-fork,karllessard/tensorflow,ppwwyyxx/tensorflow,aldian/tensorflow,karllessard/tensorflow,adit-chandra/tensorflow,aam-at/tensorflow,Intel-Corporation/tensorflow,Intel-Corporation/tensorflow,petewarden/tensorflow,chemelnucfin/tensorflow,ghchinoy/tensorflow,Bismarrck/tensorflow,tensorflow/tensorflow-pywrap_saved_model,annarev/tensorflow,freedomtan/tensorflow,jendap/tensorflow,jbedorf/tensorflow,kevin-coder/tensorflow-fork,alsrgv/tensorflow,xzturn/tensorflow,ghchinoy/tensorflow,annarev/tensorflow,jbedorf/tensorflow,ageron/tensorflow,tensorflow/tensorflow,jbedorf/tensorflow,aam-at/tensorflow,adit-chandra/tensorflow,Intel-Corporation/tensorflow,cxxgtxy/tensorflow,ageron/tensorflow,alsrgv/tensorflow,arborh/tensorflow,freedomtan/tensorflow,hfp/tensorflow-xsmm,theflofly/tensorflow,jendap/tensorflow,renyi533/tensorflow,karllessard/tensorflow,tensorflow/tensorflow,annarev/tensorflow,gautam1858/tensorflow,ageron/tensorflow,arborh/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,adit-chandra/tensorflow,Bismarrck/tensorflow,jhseu/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,kevin-coder/tensorflow-fork,jendap/tensorflow,cxxgtxy/tensorflow,tensorflow/tensorflow,adit-chandra/tensorflow,apark263/tensorflow,ppwwyyxx/tensorflow,ghchinoy/tensorflow,Intel-tensorflow/tensorflow,jhseu/tensorflow,theflofly/tensorflow,ppwwyyxx/tensorflow,alsrgv/tensorflow,frreiss/tensorflow-fred,theflofly/tensorflow,davidzchen/tensorflow,alsrgv/tensorflow,apark263/tensorflow,frreiss/tensorflow-fred,aldian/tensorflow,xzturn/tensorflow,Intel-tensorflow/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,hfp/tensorflow-xsmm,alsrgv/tensorflow,gautam1858/tensorflow,jbedorf/tensorflow,jbedorf/tensorflow,cxxgtxy/tensorflow,paolodedios/tensorflow,jendap/tensorflow,tensorflow/tensorflow-pywrap_saved_model,petewarden/tensorflow,gunan/tensorflow,aam-at/tensorflow,jbedorf/tensorflow,gunan/tensorflow,renyi533/tensorflow,adit-chandra/tensorflow,karllessard/tensorflow,ppwwyyxx/tensorflow,ghchinoy/tensorflow,xzturn/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,annarev/tensorflow,davidzchen/tensorflow,aam-at/tensorflow,tensorflow/tensorflow-pywrap_saved_model,tensorflow/tensorflow-pywrap_tf_optimizer,frreiss/tensorflow-fred,hfp/tensorflow-xsmm,adit-chandra/tensorflow,tensorflow/tensorflow,ppwwyyxx/tensorflow,annarev/tensorflow,arborh/tensorflow,renyi533/tensorflow,ageron/tensorflow,paolodedios/tensorflow,Bismarrck/tensorflow,jhseu/tensorflow,Bismarrck/tensorflow,gunan/tensorflow,yongtang/tensorflow,gautam1858/tensorflow,ageron/tensorflow,ageron/tensorflow,jbedorf/tensorflow,kevin-coder/tensorflow-fork,freedomtan/tensorflow,karllessard/tensorflow,apark263/tensorflow,adit-chandra/tensorflow,petewarden/tensorflow,jhseu/tensorflow,gautam1858/tensorflow,chemelnucfin/tensorflow,tensorflow/tensorflow-pywrap_saved_model,tensorflow/tensorflow,apark263/tensorflow,DavidNorman/tensorflow,asimshankar/tensorflow,DavidNorman/tensorflow,sarvex/tensorflow,DavidNorman/tensorflow,petewarden/tensorflow,yongtang/tensorflow,Intel-Corporation/tensorflow,ageron/tensorflow,paolodedios/tensorflow,frreiss/tensorflow-fred,tensorflow/tensorflow-pywrap_saved_model,apark263/tensorflow,ageron/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,frreiss/tensorflow-fred,theflofly/tensorflow,sarvex/tensorflow,DavidNorman/tensorflow,tensorflow/tensorflow-pywrap_saved_model,Intel-tensorflow/tensorflow,Bismarrck/tensorflow,asimshankar/tensorflow,sarvex/tensorflow,DavidNorman/tensorflow,aam-at/tensorflow,chemelnucfin/tensorflow,freedomtan/tensorflow,kevin-coder/tensorflow-fork,Intel-tensorflow/tensorflow,kevin-coder/tensorflow-fork,karllessard/tensorflow,xzturn/tensorflow,arborh/tensorflow,yongtang/tensorflow,aldian/tensorflow,Bismarrck/tensorflow,paolodedios/tensorflow,gunan/tensorflow,hfp/tensorflow-xsmm,chemelnucfin/tensorflow,petewarden/tensorflow,hfp/tensorflow-xsmm,annarev/tensorflow,chemelnucfin/tensorflow,asimshankar/tensorflow,ageron/tensorflow,kevin-coder/tensorflow-fork,paolodedios/tensorflow,Intel-tensorflow/tensorflow,aldian/tensorflow,DavidNorman/tensorflow,sarvex/tensorflow,jbedorf/tensorflow,sarvex/tensorflow,ppwwyyxx/tensorflow,chemelnucfin/tensorflow,apark263/tensorflow,annarev/tensorflow,davidzchen/tensorflow,Intel-tensorflow/tensorflow,gautam1858/tensorflow,tensorflow/tensorflow-pywrap_saved_model,yongtang/tensorflow,aam-at/tensorflow,cxxgtxy/tensorflow,Bismarrck/tensorflow,ppwwyyxx/tensorflow,jbedorf/tensorflow,adit-chandra/tensorflow,yongtang/tensorflow,jhseu/tensorflow,aldian/tensorflow,asimshankar/tensorflow,davidzchen/tensorflow,frreiss/tensorflow-fred,asimshankar/tensorflow,theflofly/tensorflow,gautam1858/tensorflow,DavidNorman/tensorflow,jendap/tensorflow,freedomtan/tensorflow,theflofly/tensorflow,tensorflow/tensorflow-pywrap_saved_model,DavidNorman/tensorflow,adit-chandra/tensorflow,ghchinoy/tensorflow,renyi533/tensorflow,adit-chandra/tensorflow,petewarden/tensorflow,davidzchen/tensorflow,hfp/tensorflow-xsmm,hfp/tensorflow-xsmm,tensorflow/tensorflow,tensorflow/tensorflow-pywrap_saved_model,petewarden/tensorflow,gautam1858/tensorflow,Bismarrck/tensorflow,arborh/tensorflow,ghchinoy/tensorflow,aam-at/tensorflow,davidzchen/tensorflow,arborh/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,alsrgv/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,ghchinoy/tensorflow,theflofly/tensorflow,xzturn/tensorflow,davidzchen/tensorflow,gautam1858/tensorflow,davidzchen/tensorflow,sarvex/tensorflow,petewarden/tensorflow,alsrgv/tensorflow,kevin-coder/tensorflow-fork,Intel-tensorflow/tensorflow,davidzchen/tensorflow,Intel-Corporation/tensorflow,asimshankar/tensorflow,annarev/tensorflow,arborh/tensorflow,ppwwyyxx/tensorflow,adit-chandra/tensorflow,freedomtan/tensorflow,freedomtan/tensorflow,DavidNorman/tensorflow,aam-at/tensorflow,aam-at/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,Intel-tensorflow/tensorflow,petewarden/tensorflow,alsrgv/tensorflow,davidzchen/tensorflow,cxxgtxy/tensorflow,asimshankar/tensorflow,xzturn/tensorflow,asimshankar/tensorflow,xzturn/tensorflow,Intel-Corporation/tensorflow,gunan/tensorflow,karllessard/tensorflow,alsrgv/tensorflow,petewarden/tensorflow,jendap/tensorflow,aam-at/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,Intel-Corporation/tensorflow,frreiss/tensorflow-fred,Bismarrck/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,yongtang/tensorflow,gautam1858/tensorflow,gunan/tensorflow,frreiss/tensorflow-fred,Intel-Corporation/tensorflow,gunan/tensorflow,jhseu/tensorflow,jbedorf/tensorflow,gunan/tensorflow,aldian/tensorflow,tensorflow/tensorflow,gunan/tensorflow,aldian/tensorflow