Protect all calls to launch cuSolver & cuBlas kernels by a lock. The code appears not to be threadsafe pre Cuda 9, and we have several report of crashes. Since the overhead is modest, better to be safe.
Protect all calls to launch cuSolver & cuBlas kernels by a lock. The code appears not to be threadsafe pre Cuda 9, and we have several report of crashes. Since the overhead is modest, better to be safe.

PiperOrigin-RevId: 179589983

AnishShah/tensorflow,jhseu/tensorflow,snnn/tensorflow,alsrgv/tensorflow,AnishShah/tensorflow,AnishShah/tensorflow,hehongliang/tensorflow,rabipanda/tensorflow,jart/tensorflow,alshedivat/tensorflow,nburn42/tensorflow,nburn42/tensorflow,alshedivat/tensorflow,xodus7/tensorflow,renyi533/tensorflow,jhseu/tensorflow,jendap/tensorflow,Xeralux/tensorflow,Intel-Corporation/tensorflow,brchiu/tensorflow,allenlavoie/tensorflow,Intel-tensorflow/tensorflow,aselle/tensorflow,adit-chandra/tensorflow,xzturn/tensorflow,zasdfgbnm/tensorflow,theflofly/tensorflow,ravindrapanda/tensorflow,girving/tensorflow,xzturn/tensorflow,asimshankar/tensorflow,yongtang/tensorflow,av8ramit/tensorflow,jendap/tensorflow,dendisuhubdy/tensorflow,Intel-tensorflow/tensorflow,tensorflow/tensorflow-pywrap_saved_model,lukeiwanski/tensorflow,Xeralux/tensorflow,adit-chandra/tensorflow,zasdfgbnm/tensorflow,meteorcloudy/tensorflow,petewarden/tensorflow,adit-chandra/tensorflow,frreiss/tensorflow-fred,eaplatanios/tensorflow,ppwwyyxx/tensorflow,frreiss/tensorflow-fred,ghchinoy/tensorflow,aselle/tensorflow,aam-at/tensorflow,zasdfgbnm/tensorflow,jalexvig/tensorflow,frreiss/tensorflow-fred,ghchinoy/tensorflow,apark263/tensorflow,meteorcloudy/tensorflow,jbedorf/tensorflow,eaplatanios/tensorflow,cxxgtxy/tensorflow,ghchinoy/tensorflow,av8ramit/tensorflow,Bismarrck/tensorflow,arborh/tensorflow,caisq/tensorflow,DavidNorman/tensorflow,jhseu/tensorflow,karllessard/tensorflow,seanli9jan/tensorflow,allenlavoie/tensorflow,snnn/tensorflow,av8ramit/tensorflow,aldian/tensorflow,hfp/tensorflow-xsmm,kevin-coder/tensorflow-fork,yongtang/tensorflow,sarvex/tensorflow,renyi533/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,xzturn/tensorflow,meteorcloudy/tensorflow,brchiu/tensorflow,theflofly/tensorflow,paolodedios/tensorflow,ppwwyyxx/tensorflow,Intel-Corporation/tensorflow,girving/tensorflow,aldian/tensorflow,ppwwyyxx/tensorflow,Bismarrck/tensorflow,sarvex/tensorflow,jart/tensorflow,jhseu/tensorflow,gautam1858/tensorflow,apark263/tensorflow,ghchinoy/tensorflow,meteorcloudy/tensorflow,ageron/tensorflow,aam-at/tensorflow,Intel-Corporation/tensorflow,chemelnucfin/tensorflow,girving/tensorflow,dancingdan/tensorflow,ghchinoy/tensorflow,nburn42/tensorflow,jalexvig/tensorflow,xzturn/tensorflow,adit-chandra/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,brchiu/tensorflow,AnishShah/tensorflow,av8ramit/tensorflow,tensorflow/tensorflow,aselle/tensorflow,nolanliou/tensorflow,theflofly/tensorflow,dongjoon-hyun/tensorflow,jendap/tensorflow,ZhangXinNan/tensorflow,av8ramit/tensorflow,theflofly/tensorflow,adit-chandra/tensorflow,kobejean/tensorflow,xodus7/tensorflow,freedomtan/tensorflow,caisq/tensorflow,caisq/tensorflow,sarvex/tensorflow,cxxgtxy/tensorflow,DavidNorman/tensorflow,yanchen036/tensorflow,aam-at/tensorflow,yongtang/tensorflow,dongjoon-hyun/tensorflow,rabipanda/tensorflow,allenlavoie/tensorflow,arborh/tensorflow,Xeralux/tensorflow,caisq/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,Intel-tensorflow/tensorflow,adit-chandra/tensorflow,adit-chandra/tensorflow,tensorflow/tensorflow,xzturn/tensorflow,jbedorf/tensorflow,Intel-tensorflow/tensorflow,ageron/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,eaplatanios/tensorflow,kevin-coder/tensorflow-fork,DavidNorman/tensorflow,aam-at/tensorflow,jalexvig/tensorflow,gojira/tensorflow,ZhangXinNan/tensorflow,arborh/tensorflow,arborh/tensorflow,adit-chandra/tensorflow,ravindrapanda/tensorflow,brchiu/tensorflow,zasdfgbnm/tensorflow,hfp/tensorflow-xsmm,tensorflow/tensorflow,kobejean/tensorflow,tensorflow/tensorflow,manipopopo/tensorflow,alsrgv/tensorflow,arborh/tensorflow,dongjoon-hyun/tensorflow,paolodedios/tensorflow,gunan/tensorflow,xzturn/tensorflow,alsrgv/tensorflow,chemelnucfin/tensorflow,freedomtan/tensorflow,petewarden/tensorflow,annarev/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,xodus7/tensorflow,drpngx/tensorflow,apark263/tensorflow,seanli9jan/tensorflow,eaplatanios/tensorflow,dendisuhubdy/tensorflow,annarev/tensorflow,renyi533/tensorflow,brchiu/tensorflow,hehongliang/tensorflow,aselle/tensorflow,theflofly/tensorflow,freedomtan/tensorflow,frreiss/tensorflow-fred,davidzchen/tensorflow,annarev/tensorflow,paolodedios/tensorflow,petewarden/tensorflow,eaplatanios/tensorflow,snnn/tensorflow,hehongliang/tensorflow,jart/tensorflow,yanchen036/tensorflow,girving/tensorflow,nolanliou/tensorflow,annarev/tensorflow,DavidNorman/tensorflow,paolodedios/tensorflow,annarev/tensorflow,kevin-coder/tensorflow-fork,aam-at/tensorflow,tensorflow/tensorflow-pywrap_saved_model,petewarden/tensorflow,Bismarrck/tensorflow,nolanliou/tensorflow,seanli9jan/tensorflow,benoitsteiner/tensorflow-xsmm,girving/tensorflow,kobejean/tensorflow,jbedorf/tensorflow,rabipanda/tensorflow,seanli9jan/tensorflow,ravindrapanda/tensorflow,av8ramit/tensorflow,gojira/tensorflow,gojira/tensorflow,tensorflow/tensorflow-pywrap_saved_model,xzturn/tensorflow,Intel-tensorflow/tensorflow,freedomtan/tensorflow,Bismarrck/tensorflow,paolodedios/tensorflow,davidzchen/tensorflow,snnn/tensorflow,xodus7/tensorflow,meteorcloudy/tensorflow,hfp/tensorflow-xsmm,DavidNorman/tensorflow,nolanliou/tensorflow,dongjoon-hyun/tensorflow,yongtang/tensorflow,gojira/tensorflow,asimshankar/tensorflow,manipopopo/tensorflow,alshedivat/tensorflow,ppwwyyxx/tensorflow,xodus7/tensorflow,yongtang/tensorflow,hfp/tensorflow-xsmm,alsrgv/tensorflow,aam-at/tensorflow,ageron/tensorflow,manipopopo/tensorflow,chemelnucfin/tensorflow,karllessard/tensorflow,gautam1858/tensorflow,gautam1858/tensorflow,rabipanda/tensorflow,yanchen036/tensorflow,karllessard/tensorflow,ZhangXinNan/tensorflow,DavidNorman/tensorflow,snnn/tensorflow,kevin-coder/tensorflow-fork,Intel-Corporation/tensorflow,karllessard/tensorflow,jbedorf/tensorflow,arborh/tensorflow,nburn42/tensorflow,Bismarrck/tensorflow,adit-chandra/tensorflow,arborh/tensorflow,paolodedios/tensorflow,nolanliou/tensorflow,jalexvig/tensorflow,freedomtan/tensorflow,ppwwyyxx/tensorflow,meteorcloudy/tensorflow,theflofly/tensorflow,apark263/tensorflow,theflofly/tensorflow,hehongliang/tensorflow,av8ramit/tensorflow,ghchinoy/tensorflow,aam-at/tensorflow,jalexvig/tensorflow,ppwwyyxx/tensorflow,gautam1858/tensorflow,lukeiwanski/tensorflow,Intel-tensorflow/tensorflow,tensorflow/tensorflow-pywrap_saved_model,chemelnucfin/tensorflow,jbedorf/tensorflow,girving/tensorflow,Bismarrck/tensorflow,snnn/tensorflow,Intel-Corporation/tensorflow,chemelnucfin/tensorflow,petewarden/tensorflow,drpngx/tensorflow,benoitsteiner/tensorflow-xsmm,av8ramit/tensorflow,yongtang/tensorflow,allenlavoie/tensorflow,zasdfgbnm/tensorflow,benoitsteiner/tensorflow-xsmm,ravindrapanda/tensorflow,alshedivat/tensorflow,karllessard/tensorflow,kevin-coder/tensorflow-fork,ghchinoy/tensorflow,cxxgtxy/tensorflow,jendap/tensorflow,aldian/tensorflow,alsrgv/tensorflow,manipopopo/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,yanchen036/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,hfp/tensorflow-xsmm,benoitsteiner/tensorflow-xsmm,jalexvig/tensorflow,aam-at/tensorflow,theflofly/tensorflow,jalexvig/tensorflow,snnn/tensorflow,manipopopo/tensorflow,dancingdan/tensorflow,dendisuhubdy/tensorflow,karllessard/tensorflow,renyi533/tensorflow,dancingdan/tensorflow,jendap/tensorflow,alsrgv/tensorflow,tensorflow/tensorflow,davidzchen/tensorflow,tensorflow/tensorflow,annarev/tensorflow,ghchinoy/tensorflow,renyi533/tensorflow,girving/tensorflow,Xeralux/tensorflow,hehongliang/tensorflow,gunan/tensorflow,nolanliou/tensorflow,DavidNorman/tensorflow,kevin-coder/tensorflow-fork,aldian/tensorflow,kevin-coder/tensorflow-fork,annarev/tensorflow,alsrgv/tensorflow,gunan/tensorflow,girving/tensorflow,ZhangXinNan/tensorflow,cxxgtxy/tensorflow,drpngx/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,ppwwyyxx/tensorflow,brchiu/tensorflow,yongtang/tensorflow,jendap/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,benoitsteiner/tensorflow-xsmm,adit-chandra/tensorflow,apark263/tensorflow,jart/tensorflow,dendisuhubdy/tensorflow,lukeiwanski/tensorflow,gautam1858/tensorflow,jart/tensorflow,gojira/tensorflow,zasdfgbnm/tensorflow,girving/tensorflow,ZhangXinNan/tensorflow,chemelnucfin/tensorflow,Xeralux/tensorflow,xodus7/tensorflow,manipopopo/tensorflow,gunan/tensorflow,nburn42/tensorflow,allenlavoie/tensorflow,apark263/tensorflow,meteorcloudy/tensorflow,ppwwyyxx/tensorflow,yongtang/tensorflow,aselle/tensorflow,dongjoon-hyun/tensorflow,AnishShah/tensorflow,jbedorf/tensorflow,jbedorf/tensorflow,yongtang/tensorflow,frreiss/tensorflow-fred,ZhangXinNan/tensorflow,ageron/tensorflow,dongjoon-hyun/tensorflow,meteorcloudy/tensorflow,asimshankar/tensorflow,Intel-Corporation/tensorflow,nburn42/tensorflow,Bismarrck/tensorflow,ravindrapanda/tensorflow,jalexvig/tensorflow,lukeiwanski/tensorflow,alsrgv/tensorflow,davidzchen/tensorflow,annarev/tensorflow,gunan/tensorflow,aselle/tensorflow,asimshankar/tensorflow,allenlavoie/tensorflow,gautam1858/tensorflow,nolanliou/tensorflow,DavidNorman/tensorflow,hehongliang/tensorflow,davidzchen/tensorflow,Intel-Corporation/tensorflow,hehongliang/tensorflow,tensorflow/tensorflow-pywrap_saved_model,hfp/tensorflow-xsmm,sarvex/tensorflow,jendap/tensorflow,gunan/tensorflow,Bismarrck/tensorflow,ageron/tensorflow,dongjoon-hyun/tensorflow,freedomtan/tensorflow,ppwwyyxx/tensorflow,dongjoon-hyun/tensorflow,theflofly/tensorflow,girving/tensorflow,tensorflow/tensorflow,jart/tensorflow,alshedivat/tensorflow,petewarden/tensorflow,ZhangXinNan/tensorflow,eaplatanios/tensorflow,drpngx/tensorflow,karllessard/tensorflow,gautam1858/tensorflow,paolodedios/tensorflow,xodus7/tensorflow,benoitsteiner/tensorflow-xsmm,benoitsteiner/tensorflow-xsmm,ppwwyyxx/tensorflow,dancingdan/tensorflow,asimshankar/tensorflow,brchiu/tensorflow,jart/tensorflow,tensorflow/tensorflow,cxxgtxy/tensorflow,asimshankar/tensorflow,ageron/tensorflow,aldian/tensorflow,adit-chandra/tensorflow,petewarden/tensorflow,freedomtan/tensorflow,lukeiwanski/tensorflow,Intel-Corporation/tensorflow,av8ramit/tensorflow,eaplatanios/tensorflow,dancingdan/tensorflow,xzturn/tensorflow,brchiu/tensorflow,sarvex/tensorflow,Intel-tensorflow/tensorflow,rabipanda/tensorflow,dendisuhubdy/tensorflow,alsrgv/tensorflow,cxxgtxy/tensorflow,manipopopo/tensorflow,xodus7/tensorflow,jalexvig/tensorflow,paolodedios/tensorflow,aam-at/tensorflow,Xeralux/tensorflow,eaplatanios/tensorflow,sarvex/tensorflow,jhseu/tensorflow,Xeralux/tensorflow,girving/tensorflow,nolanliou/tensorflow,dancingdan/tensorflow,aselle/tensorflow,Xeralux/tensorflow,kobejean/tensorflow,snnn/tensorflow,tensorflow/tensorflow-pywrap_saved_model,jart/tensorflow,xzturn/tensorflow,brchiu/tensorflow,hfp/tensorflow-xsmm,caisq/tensorflow,lukeiwanski/tensorflow,jart/tensorflow,dendisuhubdy/tensorflow,jhseu/tensorflow,gunan/tensorflow,gojira/tensorflow,meteorcloudy/tensorflow,petewarden/tensorflow,Bismarrck/tensorflow,davidzchen/tensorflow,gautam1858/tensorflow,DavidNorman/tensorflow,drpngx/tensorflow,cxxgtxy/tensorflow,dancingdan/tensorflow,tensorflow/tensorflow,alshedivat/tensorflow,dongjoon-hyun/tensorflow,allenlavoie/tensorflow,ZhangXinNan/tensorflow,dendisuhubdy/tensorflow,dancingdan/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,kobejean/tensorflow,seanli9jan/tensorflow,caisq/tensorflow,asimshankar/tensorflow,aldian/tensorflow,jhseu/tensorflow,caisq/tensorflow,frreiss/tensorflow-fred,caisq/tensorflow,gojira/tensorflow,drpngx/tensorflow,seanli9jan/tensorflow,kobejean/tensorflow,aselle/tensorflow,gunan/tensorflow,snnn/tensorflow,kobejean/tensorflow,rabipanda/tensorflow,freedomtan/tensorflow,davidzchen/tensorflow,AnishShah/tensorflow,arborh/tensorflow,tensorflow/tensorflow,sarvex/tensorflow,arborh/tensorflow,Intel-tensorflow/tensorflow,frreiss/tensorflow-fred,yanchen036/tensorflow,kevin-coder/tensorflow-fork,petewarden/tensorflow,gojira/tensorflow,ravindrapanda/tensorflow,brchiu/tensorflow,yanchen036/tensorflow,karllessard/tensorflow,tensorflow/tensorflow-pywrap_saved_model,Bismarrck/tensorflow,alshedivat/tensorflow,ZhangXinNan/tensorflow,paolodedios/tensorflow,dendisuhubdy/tensorflow,yongtang/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,annarev/tensorflow,gunan/tensorflow,karllessard/tensorflow,arborh/tensorflow,nburn42/tensorflow,tensorflow/tensorflow-pywrap_saved_model,kobejean/tensorflow,tensorflow/tensorflow-pywrap_saved_model,asimshankar/tensorflow,gautam1858/tensorflow,xodus7/tensorflow,Xeralux/tensorflow,jbedorf/tensorflow,yongtang/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,gautam1858/tensorflow,jhseu/tensorflow,nburn42/tensorflow,ppwwyyxx/tensorflow,renyi533/tensorflow,davidzchen/tensorflow,ppwwyyxx/tensorflow,AnishShah/tensorflow,Bismarrck/tensorflow,manipopopo/tensorflow,hfp/tensorflow-xsmm,arborh/tensorflow,adit-chandra/tensorflow,asimshankar/tensorflow,snnn/tensorflow,Intel-tensorflow/tensorflow,allenlavoie/tensorflow,karllessard/tensorflow,seanli9jan/tensorflow,seanli9jan/tensorflow,tensorflow/tensorflow-pywrap_saved_model,ZhangXinNan/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,eaplatanios/tensorflow,paolodedios/tensorflow,paolodedios/tensorflow,aldian/tensorflow,av8ramit/tensorflow,apark263/tensorflow,manipopopo/tensorflow,benoitsteiner/tensorflow-xsmm,freedomtan/tensorflow,lukeiwanski/tensorflow,av8ramit/tensorflow,jhseu/tensorflow,nburn42/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,ghchinoy/tensorflow,jbedorf/tensorflow,yanchen036/tensorflow,frreiss/tensorflow-fred,nburn42/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,dongjoon-hyun/tensorflow,freedomtan/tensorflow,seanli9jan/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,nburn42/tensorflow,renyi533/tensorflow,chemelnucfin/tensorflow,freedomtan/tensorflow,annarev/tensorflow,dendisuhubdy/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,zasdfgbnm/tensorflow,jbedorf/tensorflow,ageron/tensorflow,gautam1858/tensorflow,allenlavoie/tensorflow,alsrgv/tensorflow,xzturn/tensorflow,Intel-tensorflow/tensorflow,lukeiwanski/tensorflow,caisq/tensorflow,alsrgv/tensorflow,freedomtan/tensorflow,yanchen036/tensorflow,ZhangXinNan/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,AnishShah/tensorflow,benoitsteiner/tensorflow-xsmm,chemelnucfin/tensorflow,dongjoon-hyun/tensorflow,eaplatanios/tensorflow,seanli9jan/tensorflow,allenlavoie/tensorflow,gojira/tensorflow,kobejean/tensorflow,cxxgtxy/tensorflow,dancingdan/tensorflow,kevin-coder/tensorflow-fork,lukeiwanski/tensorflow,jart/tensorflow,Xeralux/tensorflow,rabipanda/tensorflow,nolanliou/tensorflow,AnishShah/tensorflow,asimshankar/tensorflow,apark263/tensorflow,chemelnucfin/tensorflow,aam-at/tensorflow,Intel-tensorflow/tensorflow,meteorcloudy/tensorflow,manipopopo/tensorflow,xzturn/tensorflow,renyi533/tensorflow,aselle/tensorflow,ravindrapanda/tensorflow,xzturn/tensorflow,DavidNorman/tensorflow,zasdfgbnm/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,jbedorf/tensorflow,tensorflow/tensorflow,alshedivat/tensorflow,kevin-coder/tensorflow-fork,aselle/tensorflow,drpngx/tensorflow,Xeralux/tensorflow,frreiss/tensorflow-fred,DavidNorman/tensorflow,gautam1858/tensorflow,apark263/tensorflow,ageron/tensorflow,aldian/tensorflow,jalexvig/tensorflow,lukeiwanski/tensorflow,kevin-coder/tensorflow-fork,aselle/tensorflow,kobejean/tensorflow,davidzchen/tensorflow,jendap/tensorflow,eaplatanios/tensorflow,petewarden/tensorflow,aam-at/tensorflow,brchiu/tensorflow,zasdfgbnm/tensorflow,alshedivat/tensorflow,renyi533/tensorflow,frreiss/tensorflow-fred,nolanliou/tensorflow,rabipanda/tensorflow,petewarden/tensorflow,xodus7/tensorflow,frreiss/tensorflow-fred,frreiss/tensorflow-fred,benoitsteiner/tensorflow-xsmm,dancingdan/tensorflow,drpngx/tensorflow,ageron/tensorflow,gojira/tensorflow,jhseu/tensorflow,ghchinoy/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,drpngx/tensorflow,tensorflow/tensorflow-pywrap_saved_model,rabipanda/tensorflow,ravindrapanda/tensorflow,seanli9jan/tensorflow,jendap/tensorflow,kobejean/tensorflow,jhseu/tensorflow,chemelnucfin/tensorflow,renyi533/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,ageron/tensorflow,aam-at/tensorflow,zasdfgbnm/tensorflow,AnishShah/tensorflow,jendap/tensorflow,gunan/tensorflow,chemelnucfin/tensorflow,AnishShah/tensorflow,jendap/tensorflow,gunan/tensorflow,alshedivat/tensorflow,theflofly/tensorflow,allenlavoie/tensorflow,alsrgv/tensorflow,apark263/tensorflow,hfp/tensorflow-xsmm,jhseu/tensorflow,arborh/tensorflow,alshedivat/tensorflow,jbedorf/tensorflow,manipopopo/tensorflow,ageron/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,xodus7/tensorflow,gunan/tensorflow,snnn/tensorflow,ravindrapanda/tensorflow,asimshankar/tensorflow,davidzchen/tensorflow,dancingdan/tensorflow,jalexvig/tensorflow,renyi533/tensorflow,gojira/tensorflow,ghchinoy/tensorflow,dendisuhubdy/tensorflow,ageron/tensorflow,hfp/tensorflow-xsmm,theflofly/tensorflow,sarvex/tensorflow,davidzchen/tensorflow,davidzchen/tensorflow,karllessard/tensorflow,hfp/tensorflow-xsmm,benoitsteiner/tensorflow-xsmm,petewarden/tensorflow,caisq/tensorflow,annarev/tensorflow,rabipanda/tensorflow,drpngx/tensorflow,apark263/tensorflow,zasdfgbnm/tensorflow,renyi533/tensorflow,chemelnucfin/tensorflow,ghchinoy/tensorflow,DavidNorman/tensorflow,theflofly/tensorflow,rabipanda/tensorflow