Do not block inside NCCL OpKernel when computing all-sum.
Do not block inside NCCL OpKernel when computing all-sum.

Collective ops execute NCCL all-reduce in 2 stages.  The first step executes
the all-sum or all-product, and the second step executes the final operation -
identity or divide by group size.  The final op is useful if, e.g., the user
would like to compute the mean of all inputs.

The currently implementation blocks the thread of execution until the all-sum
is done, and then executes the final op.  However, in the case that the user
wants to compute all-sum, which is fairly common, the blocking is unnecessary.
This change removes the unnecessary blocking.

PiperOrigin-RevId: 268968162

arborh/tensorflow,aam-at/tensorflow,tensorflow/tensorflow-pywrap_saved_model,gunan/tensorflow,frreiss/tensorflow-fred,xzturn/tensorflow,paolodedios/tensorflow,freedomtan/tensorflow,jhseu/tensorflow,Intel-Corporation/tensorflow,frreiss/tensorflow-fred,yongtang/tensorflow,ppwwyyxx/tensorflow,xzturn/tensorflow,petewarden/tensorflow,aldian/tensorflow,annarev/tensorflow,xzturn/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,renyi533/tensorflow,chemelnucfin/tensorflow,frreiss/tensorflow-fred,petewarden/tensorflow,gunan/tensorflow,aldian/tensorflow,arborh/tensorflow,gunan/tensorflow,DavidNorman/tensorflow,gunan/tensorflow,cxxgtxy/tensorflow,xzturn/tensorflow,jhseu/tensorflow,xzturn/tensorflow,DavidNorman/tensorflow,tensorflow/tensorflow-pywrap_saved_model,paolodedios/tensorflow,gautam1858/tensorflow,adit-chandra/tensorflow,arborh/tensorflow,renyi533/tensorflow,gunan/tensorflow,tensorflow/tensorflow,tensorflow/tensorflow,Intel-Corporation/tensorflow,petewarden/tensorflow,annarev/tensorflow,frreiss/tensorflow-fred,aam-at/tensorflow,cxxgtxy/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,ppwwyyxx/tensorflow,tensorflow/tensorflow-pywrap_saved_model,frreiss/tensorflow-fred,renyi533/tensorflow,ppwwyyxx/tensorflow,gautam1858/tensorflow,chemelnucfin/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,DavidNorman/tensorflow,tensorflow/tensorflow-pywrap_saved_model,tensorflow/tensorflow-pywrap_saved_model,aam-at/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,sarvex/tensorflow,chemelnucfin/tensorflow,Intel-tensorflow/tensorflow,ppwwyyxx/tensorflow,gautam1858/tensorflow,freedomtan/tensorflow,gunan/tensorflow,jhseu/tensorflow,gautam1858/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,arborh/tensorflow,adit-chandra/tensorflow,tensorflow/tensorflow-pywrap_saved_model,adit-chandra/tensorflow,tensorflow/tensorflow,jhseu/tensorflow,freedomtan/tensorflow,Intel-tensorflow/tensorflow,tensorflow/tensorflow,DavidNorman/tensorflow,tensorflow/tensorflow,karllessard/tensorflow,freedomtan/tensorflow,arborh/tensorflow,gunan/tensorflow,chemelnucfin/tensorflow,gautam1858/tensorflow,jhseu/tensorflow,yongtang/tensorflow,aam-at/tensorflow,davidzchen/tensorflow,renyi533/tensorflow,ppwwyyxx/tensorflow,tensorflow/tensorflow,jhseu/tensorflow,adit-chandra/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,sarvex/tensorflow,yongtang/tensorflow,gunan/tensorflow,paolodedios/tensorflow,adit-chandra/tensorflow,annarev/tensorflow,arborh/tensorflow,aam-at/tensorflow,ppwwyyxx/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,adit-chandra/tensorflow,sarvex/tensorflow,adit-chandra/tensorflow,davidzchen/tensorflow,frreiss/tensorflow-fred,xzturn/tensorflow,davidzchen/tensorflow,jhseu/tensorflow,aam-at/tensorflow,xzturn/tensorflow,DavidNorman/tensorflow,aldian/tensorflow,DavidNorman/tensorflow,freedomtan/tensorflow,gautam1858/tensorflow,tensorflow/tensorflow-pywrap_saved_model,arborh/tensorflow,karllessard/tensorflow,tensorflow/tensorflow-pywrap_saved_model,frreiss/tensorflow-fred,freedomtan/tensorflow,annarev/tensorflow,yongtang/tensorflow,Intel-tensorflow/tensorflow,aldian/tensorflow,Intel-Corporation/tensorflow,freedomtan/tensorflow,sarvex/tensorflow,xzturn/tensorflow,arborh/tensorflow,gautam1858/tensorflow,Intel-tensorflow/tensorflow,karllessard/tensorflow,Intel-Corporation/tensorflow,jhseu/tensorflow,karllessard/tensorflow,ppwwyyxx/tensorflow,aldian/tensorflow,karllessard/tensorflow,chemelnucfin/tensorflow,adit-chandra/tensorflow,chemelnucfin/tensorflow,Intel-tensorflow/tensorflow,chemelnucfin/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,tensorflow/tensorflow-pywrap_tf_optimizer,gautam1858/tensorflow,petewarden/tensorflow,gautam1858/tensorflow,annarev/tensorflow,davidzchen/tensorflow,adit-chandra/tensorflow,paolodedios/tensorflow,adit-chandra/tensorflow,Intel-Corporation/tensorflow,Intel-tensorflow/tensorflow,tensorflow/tensorflow,renyi533/tensorflow,Intel-Corporation/tensorflow,chemelnucfin/tensorflow,annarev/tensorflow,xzturn/tensorflow,chemelnucfin/tensorflow,frreiss/tensorflow-fred,paolodedios/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,tensorflow/tensorflow-pywrap_saved_model,Intel-tensorflow/tensorflow,gunan/tensorflow,frreiss/tensorflow-fred,yongtang/tensorflow,yongtang/tensorflow,davidzchen/tensorflow,cxxgtxy/tensorflow,petewarden/tensorflow,ppwwyyxx/tensorflow,gunan/tensorflow,sarvex/tensorflow,petewarden/tensorflow,paolodedios/tensorflow,ppwwyyxx/tensorflow,yongtang/tensorflow,aam-at/tensorflow,annarev/tensorflow,freedomtan/tensorflow,sarvex/tensorflow,renyi533/tensorflow,davidzchen/tensorflow,ppwwyyxx/tensorflow,petewarden/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,ppwwyyxx/tensorflow,freedomtan/tensorflow,paolodedios/tensorflow,paolodedios/tensorflow,DavidNorman/tensorflow,karllessard/tensorflow,xzturn/tensorflow,karllessard/tensorflow,cxxgtxy/tensorflow,gunan/tensorflow,Intel-tensorflow/tensorflow,chemelnucfin/tensorflow,tensorflow/tensorflow-pywrap_saved_model,renyi533/tensorflow,davidzchen/tensorflow,cxxgtxy/tensorflow,chemelnucfin/tensorflow,annarev/tensorflow,karllessard/tensorflow,gautam1858/tensorflow,tensorflow/tensorflow,jhseu/tensorflow,davidzchen/tensorflow,Intel-Corporation/tensorflow,freedomtan/tensorflow,aldian/tensorflow,DavidNorman/tensorflow,davidzchen/tensorflow,frreiss/tensorflow-fred,aam-at/tensorflow,karllessard/tensorflow,karllessard/tensorflow,petewarden/tensorflow,aldian/tensorflow,yongtang/tensorflow,chemelnucfin/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,Intel-tensorflow/tensorflow,annarev/tensorflow,davidzchen/tensorflow,aam-at/tensorflow,davidzchen/tensorflow,yongtang/tensorflow,freedomtan/tensorflow,tensorflow/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,tensorflow/tensorflow-experimental_link_static_libraries_once,davidzchen/tensorflow,arborh/tensorflow,jhseu/tensorflow,renyi533/tensorflow,Intel-Corporation/tensorflow,paolodedios/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,renyi533/tensorflow,DavidNorman/tensorflow,jhseu/tensorflow,adit-chandra/tensorflow,frreiss/tensorflow-fred,tensorflow/tensorflow-pywrap_tf_optimizer,aam-at/tensorflow,renyi533/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,tensorflow/tensorflow,arborh/tensorflow,renyi533/tensorflow,paolodedios/tensorflow,xzturn/tensorflow,renyi533/tensorflow,freedomtan/tensorflow,cxxgtxy/tensorflow,gunan/tensorflow,gautam1858/tensorflow,DavidNorman/tensorflow,tensorflow/tensorflow-pywrap_saved_model,arborh/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,frreiss/tensorflow-fred,DavidNorman/tensorflow,cxxgtxy/tensorflow,petewarden/tensorflow,petewarden/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,aam-at/tensorflow,petewarden/tensorflow,Intel-tensorflow/tensorflow,petewarden/tensorflow,aam-at/tensorflow,DavidNorman/tensorflow,ppwwyyxx/tensorflow,arborh/tensorflow,tensorflow/tensorflow,annarev/tensorflow,annarev/tensorflow,Intel-tensorflow/tensorflow,sarvex/tensorflow,aldian/tensorflow,paolodedios/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,yongtang/tensorflow,xzturn/tensorflow,cxxgtxy/tensorflow,jhseu/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,tensorflow/tensorflow-pywrap_tf_optimizer,karllessard/tensorflow,gautam1858/tensorflow,adit-chandra/tensorflow,yongtang/tensorflow,sarvex/tensorflow