Change the Eigen reduction code to use a tree to improve numerical stability. This changes the InnerMostDimReducer to use a summation tree, which is more numerically stable than the previous approach of sequential addition into an accumulator. This solves the issue for reduction over all or a trailing subset of dimensions. This change does not improve the numerical accuracy for MeanReducer, which maintains state.
Change the Eigen reduction code to use a tree to improve numerical stability.
This changes the InnerMostDimReducer to use a summation tree, which is more numerically stable than the previous approach of sequential addition into an accumulator.
This solves the issue for reduction over all or a trailing subset of dimensions.
This change does not improve the numerical accuracy for MeanReducer, which maintains state.

Benchmarks show a 40% (AVX) to 50% (SSE) slowdown for small row reductions (sum, float). column- and full reductions are unchanged.

Cleaned up TensorFunctors.h a bit by moving the traits to reducer_traits and updating the code that uses the reducers accordingly.

Introduced a new trait "IsExactlyAssociative" and new template specializations of InnerMostDimReducer to ensure that we only invoke the new and slightly more expensive codepath when it is needed, i.e. for sum reduction of non-integer types.

PiperOrigin-RevId: 198946075

ZhangXinNan/tensorflow,xodus7/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,xzturn/tensorflow,Bismarrck/tensorflow,tensorflow/tensorflow-pywrap_saved_model,tensorflow/tensorflow-pywrap_tf_optimizer,girving/tensorflow,jbedorf/tensorflow,petewarden/tensorflow,drpngx/tensorflow,hehongliang/tensorflow,gojira/tensorflow,lukeiwanski/tensorflow,yongtang/tensorflow,kobejean/tensorflow,gautam1858/tensorflow,hfp/tensorflow-xsmm,ppwwyyxx/tensorflow,snnn/tensorflow,jalexvig/tensorflow,dancingdan/tensorflow,ageron/tensorflow,seanli9jan/tensorflow,dendisuhubdy/tensorflow,frreiss/tensorflow-fred,gautam1858/tensorflow,apark263/tensorflow,apark263/tensorflow,xzturn/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,theflofly/tensorflow,ZhangXinNan/tensorflow,snnn/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,yongtang/tensorflow,karllessard/tensorflow,jhseu/tensorflow,jbedorf/tensorflow,sarvex/tensorflow,dendisuhubdy/tensorflow,arborh/tensorflow,xodus7/tensorflow,benoitsteiner/tensorflow-xsmm,aselle/tensorflow,petewarden/tensorflow,tensorflow/tensorflow-pywrap_saved_model,jalexvig/tensorflow,dongjoon-hyun/tensorflow,gautam1858/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,davidzchen/tensorflow,kobejean/tensorflow,brchiu/tensorflow,frreiss/tensorflow-fred,caisq/tensorflow,girving/tensorflow,karllessard/tensorflow,jbedorf/tensorflow,kevin-coder/tensorflow-fork,meteorcloudy/tensorflow,xzturn/tensorflow,Intel-tensorflow/tensorflow,frreiss/tensorflow-fred,karllessard/tensorflow,jhseu/tensorflow,jhseu/tensorflow,dongjoon-hyun/tensorflow,gautam1858/tensorflow,jendap/tensorflow,jbedorf/tensorflow,apark263/tensorflow,Intel-Corporation/tensorflow,kobejean/tensorflow,frreiss/tensorflow-fred,jart/tensorflow,apark263/tensorflow,tensorflow/tensorflow,theflofly/tensorflow,arborh/tensorflow,tensorflow/tensorflow,chemelnucfin/tensorflow,Intel-Corporation/tensorflow,paolodedios/tensorflow,paolodedios/tensorflow,snnn/tensorflow,cxxgtxy/tensorflow,brchiu/tensorflow,dongjoon-hyun/tensorflow,asimshankar/tensorflow,annarev/tensorflow,paolodedios/tensorflow,dendisuhubdy/tensorflow,gunan/tensorflow,aselle/tensorflow,manipopopo/tensorflow,ghchinoy/tensorflow,jhseu/tensorflow,yanchen036/tensorflow,jendap/tensorflow,arborh/tensorflow,jart/tensorflow,ZhangXinNan/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,aam-at/tensorflow,lukeiwanski/tensorflow,benoitsteiner/tensorflow-xsmm,tensorflow/tensorflow,apark263/tensorflow,dongjoon-hyun/tensorflow,karllessard/tensorflow,alsrgv/tensorflow,girving/tensorflow,davidzchen/tensorflow,asimshankar/tensorflow,tensorflow/tensorflow-pywrap_saved_model,aselle/tensorflow,dongjoon-hyun/tensorflow,adit-chandra/tensorflow,renyi533/tensorflow,chemelnucfin/tensorflow,cxxgtxy/tensorflow,kobejean/tensorflow,kevin-coder/tensorflow-fork,ageron/tensorflow,drpngx/tensorflow,jhseu/tensorflow,karllessard/tensorflow,jendap/tensorflow,frreiss/tensorflow-fred,kevin-coder/tensorflow-fork,chemelnucfin/tensorflow,jhseu/tensorflow,hehongliang/tensorflow,ZhangXinNan/tensorflow,dendisuhubdy/tensorflow,freedomtan/tensorflow,jbedorf/tensorflow,jendap/tensorflow,davidzchen/tensorflow,asimshankar/tensorflow,Intel-tensorflow/tensorflow,apark263/tensorflow,frreiss/tensorflow-fred,drpngx/tensorflow,davidzchen/tensorflow,petewarden/tensorflow,petewarden/tensorflow,Bismarrck/tensorflow,ghchinoy/tensorflow,arborh/tensorflow,frreiss/tensorflow-fred,ageron/tensorflow,brchiu/tensorflow,yongtang/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,dongjoon-hyun/tensorflow,dancingdan/tensorflow,dancingdan/tensorflow,gautam1858/tensorflow,kevin-coder/tensorflow-fork,girving/tensorflow,hfp/tensorflow-xsmm,gunan/tensorflow,yongtang/tensorflow,adit-chandra/tensorflow,hehongliang/tensorflow,freedomtan/tensorflow,hfp/tensorflow-xsmm,ppwwyyxx/tensorflow,tensorflow/tensorflow,gunan/tensorflow,jhseu/tensorflow,DavidNorman/tensorflow,kevin-coder/tensorflow-fork,theflofly/tensorflow,ppwwyyxx/tensorflow,chemelnucfin/tensorflow,Bismarrck/tensorflow,adit-chandra/tensorflow,adit-chandra/tensorflow,hehongliang/tensorflow,alshedivat/tensorflow,aam-at/tensorflow,ghchinoy/tensorflow,lukeiwanski/tensorflow,gautam1858/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,dancingdan/tensorflow,davidzchen/tensorflow,renyi533/tensorflow,kobejean/tensorflow,manipopopo/tensorflow,jart/tensorflow,yanchen036/tensorflow,cxxgtxy/tensorflow,Intel-tensorflow/tensorflow,Intel-tensorflow/tensorflow,yongtang/tensorflow,adit-chandra/tensorflow,jbedorf/tensorflow,gunan/tensorflow,jbedorf/tensorflow,kevin-coder/tensorflow-fork,jalexvig/tensorflow,xodus7/tensorflow,arborh/tensorflow,ageron/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,renyi533/tensorflow,lukeiwanski/tensorflow,tensorflow/tensorflow-pywrap_saved_model,Intel-Corporation/tensorflow,adit-chandra/tensorflow,manipopopo/tensorflow,theflofly/tensorflow,karllessard/tensorflow,Intel-tensorflow/tensorflow,apark263/tensorflow,tensorflow/tensorflow-pywrap_saved_model,ZhangXinNan/tensorflow,sarvex/tensorflow,arborh/tensorflow,chemelnucfin/tensorflow,renyi533/tensorflow,aam-at/tensorflow,renyi533/tensorflow,drpngx/tensorflow,xodus7/tensorflow,seanli9jan/tensorflow,ageron/tensorflow,benoitsteiner/tensorflow-xsmm,kobejean/tensorflow,Bismarrck/tensorflow,kobejean/tensorflow,ZhangXinNan/tensorflow,davidzchen/tensorflow,manipopopo/tensorflow,xzturn/tensorflow,annarev/tensorflow,lukeiwanski/tensorflow,karllessard/tensorflow,ppwwyyxx/tensorflow,caisq/tensorflow,aldian/tensorflow,freedomtan/tensorflow,Intel-Corporation/tensorflow,annarev/tensorflow,renyi533/tensorflow,jhseu/tensorflow,aldian/tensorflow,alshedivat/tensorflow,hfp/tensorflow-xsmm,jendap/tensorflow,xzturn/tensorflow,manipopopo/tensorflow,drpngx/tensorflow,ppwwyyxx/tensorflow,kobejean/tensorflow,adit-chandra/tensorflow,jart/tensorflow,hfp/tensorflow-xsmm,seanli9jan/tensorflow,DavidNorman/tensorflow,Intel-tensorflow/tensorflow,frreiss/tensorflow-fred,seanli9jan/tensorflow,Bismarrck/tensorflow,ageron/tensorflow,benoitsteiner/tensorflow-xsmm,apark263/tensorflow,ZhangXinNan/tensorflow,jalexvig/tensorflow,chemelnucfin/tensorflow,meteorcloudy/tensorflow,tensorflow/tensorflow-pywrap_saved_model,petewarden/tensorflow,dendisuhubdy/tensorflow,xzturn/tensorflow,caisq/tensorflow,petewarden/tensorflow,aselle/tensorflow,manipopopo/tensorflow,jalexvig/tensorflow,ghchinoy/tensorflow,freedomtan/tensorflow,annarev/tensorflow,seanli9jan/tensorflow,hehongliang/tensorflow,ZhangXinNan/tensorflow,gautam1858/tensorflow,jart/tensorflow,manipopopo/tensorflow,davidzchen/tensorflow,gojira/tensorflow,theflofly/tensorflow,gojira/tensorflow,girving/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,jalexvig/tensorflow,alsrgv/tensorflow,davidzchen/tensorflow,ppwwyyxx/tensorflow,tensorflow/tensorflow,chemelnucfin/tensorflow,sarvex/tensorflow,Intel-Corporation/tensorflow,gautam1858/tensorflow,snnn/tensorflow,yanchen036/tensorflow,drpngx/tensorflow,xodus7/tensorflow,petewarden/tensorflow,meteorcloudy/tensorflow,alshedivat/tensorflow,asimshankar/tensorflow,manipopopo/tensorflow,jbedorf/tensorflow,lukeiwanski/tensorflow,benoitsteiner/tensorflow-xsmm,benoitsteiner/tensorflow-xsmm,brchiu/tensorflow,jart/tensorflow,meteorcloudy/tensorflow,xodus7/tensorflow,snnn/tensorflow,benoitsteiner/tensorflow-xsmm,drpngx/tensorflow,alsrgv/tensorflow,ghchinoy/tensorflow,aselle/tensorflow,gojira/tensorflow,freedomtan/tensorflow,petewarden/tensorflow,aselle/tensorflow,alsrgv/tensorflow,jendap/tensorflow,snnn/tensorflow,arborh/tensorflow,gautam1858/tensorflow,xzturn/tensorflow,caisq/tensorflow,theflofly/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,Intel-tensorflow/tensorflow,tensorflow/tensorflow-pywrap_saved_model,benoitsteiner/tensorflow-xsmm,davidzchen/tensorflow,apark263/tensorflow,karllessard/tensorflow,arborh/tensorflow,karllessard/tensorflow,Intel-Corporation/tensorflow,AnishShah/tensorflow,kevin-coder/tensorflow-fork,tensorflow/tensorflow-experimental_link_static_libraries_once,frreiss/tensorflow-fred,dendisuhubdy/tensorflow,ghchinoy/tensorflow,DavidNorman/tensorflow,AnishShah/tensorflow,ageron/tensorflow,paolodedios/tensorflow,ppwwyyxx/tensorflow,aam-at/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,ghchinoy/tensorflow,annarev/tensorflow,dongjoon-hyun/tensorflow,ageron/tensorflow,xodus7/tensorflow,jart/tensorflow,yanchen036/tensorflow,alsrgv/tensorflow,cxxgtxy/tensorflow,aam-at/tensorflow,ppwwyyxx/tensorflow,Bismarrck/tensorflow,brchiu/tensorflow,jalexvig/tensorflow,meteorcloudy/tensorflow,cxxgtxy/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,theflofly/tensorflow,jart/tensorflow,drpngx/tensorflow,ppwwyyxx/tensorflow,jendap/tensorflow,gautam1858/tensorflow,alsrgv/tensorflow,xzturn/tensorflow,alshedivat/tensorflow,chemelnucfin/tensorflow,AnishShah/tensorflow,ghchinoy/tensorflow,aldian/tensorflow,meteorcloudy/tensorflow,petewarden/tensorflow,annarev/tensorflow,DavidNorman/tensorflow,kobejean/tensorflow,gunan/tensorflow,aldian/tensorflow,hfp/tensorflow-xsmm,yongtang/tensorflow,Intel-tensorflow/tensorflow,renyi533/tensorflow,xzturn/tensorflow,theflofly/tensorflow,DavidNorman/tensorflow,freedomtan/tensorflow,annarev/tensorflow,ppwwyyxx/tensorflow,annarev/tensorflow,snnn/tensorflow,asimshankar/tensorflow,benoitsteiner/tensorflow-xsmm,alshedivat/tensorflow,brchiu/tensorflow,brchiu/tensorflow,theflofly/tensorflow,sarvex/tensorflow,AnishShah/tensorflow,yongtang/tensorflow,freedomtan/tensorflow,girving/tensorflow,AnishShah/tensorflow,jart/tensorflow,gojira/tensorflow,dancingdan/tensorflow,tensorflow/tensorflow,apark263/tensorflow,aldian/tensorflow,jbedorf/tensorflow,seanli9jan/tensorflow,yanchen036/tensorflow,gautam1858/tensorflow,theflofly/tensorflow,dancingdan/tensorflow,aam-at/tensorflow,DavidNorman/tensorflow,tensorflow/tensorflow-pywrap_saved_model,sarvex/tensorflow,AnishShah/tensorflow,benoitsteiner/tensorflow-xsmm,jhseu/tensorflow,alshedivat/tensorflow,alshedivat/tensorflow,Bismarrck/tensorflow,xzturn/tensorflow,alshedivat/tensorflow,kobejean/tensorflow,asimshankar/tensorflow,Intel-tensorflow/tensorflow,DavidNorman/tensorflow,hehongliang/tensorflow,gunan/tensorflow,gojira/tensorflow,hfp/tensorflow-xsmm,ageron/tensorflow,ageron/tensorflow,aselle/tensorflow,asimshankar/tensorflow,aam-at/tensorflow,Bismarrck/tensorflow,theflofly/tensorflow,girving/tensorflow,chemelnucfin/tensorflow,snnn/tensorflow,benoitsteiner/tensorflow-xsmm,Bismarrck/tensorflow,ageron/tensorflow,gojira/tensorflow,alshedivat/tensorflow,caisq/tensorflow,hehongliang/tensorflow,renyi533/tensorflow,jendap/tensorflow,AnishShah/tensorflow,renyi533/tensorflow,paolodedios/tensorflow,aselle/tensorflow,arborh/tensorflow,annarev/tensorflow,ppwwyyxx/tensorflow,drpngx/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,girving/tensorflow,snnn/tensorflow,lukeiwanski/tensorflow,asimshankar/tensorflow,sarvex/tensorflow,ZhangXinNan/tensorflow,davidzchen/tensorflow,snnn/tensorflow,jhseu/tensorflow,meteorcloudy/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,DavidNorman/tensorflow,adit-chandra/tensorflow,dendisuhubdy/tensorflow,cxxgtxy/tensorflow,asimshankar/tensorflow,gunan/tensorflow,kobejean/tensorflow,gojira/tensorflow,dendisuhubdy/tensorflow,paolodedios/tensorflow,girving/tensorflow,freedomtan/tensorflow,dancingdan/tensorflow,freedomtan/tensorflow,freedomtan/tensorflow,yanchen036/tensorflow,ghchinoy/tensorflow,yongtang/tensorflow,hfp/tensorflow-xsmm,petewarden/tensorflow,xzturn/tensorflow,caisq/tensorflow,alsrgv/tensorflow,DavidNorman/tensorflow,dancingdan/tensorflow,asimshankar/tensorflow,aldian/tensorflow,cxxgtxy/tensorflow,frreiss/tensorflow-fred,gautam1858/tensorflow,theflofly/tensorflow,jbedorf/tensorflow,jalexvig/tensorflow,jendap/tensorflow,kevin-coder/tensorflow-fork,renyi533/tensorflow,manipopopo/tensorflow,AnishShah/tensorflow,alshedivat/tensorflow,Intel-Corporation/tensorflow,yongtang/tensorflow,Bismarrck/tensorflow,karllessard/tensorflow,caisq/tensorflow,kevin-coder/tensorflow-fork,gojira/tensorflow,renyi533/tensorflow,manipopopo/tensorflow,meteorcloudy/tensorflow,gunan/tensorflow,tensorflow/tensorflow-pywrap_saved_model,davidzchen/tensorflow,xodus7/tensorflow,manipopopo/tensorflow,sarvex/tensorflow,hfp/tensorflow-xsmm,adit-chandra/tensorflow,arborh/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,hfp/tensorflow-xsmm,adit-chandra/tensorflow,chemelnucfin/tensorflow,aam-at/tensorflow,dancingdan/tensorflow,gojira/tensorflow,jendap/tensorflow,seanli9jan/tensorflow,annarev/tensorflow,kevin-coder/tensorflow-fork,brchiu/tensorflow,paolodedios/tensorflow,xodus7/tensorflow,jendap/tensorflow,paolodedios/tensorflow,jalexvig/tensorflow,annarev/tensorflow,gunan/tensorflow,xodus7/tensorflow,Bismarrck/tensorflow,aam-at/tensorflow,ZhangXinNan/tensorflow,meteorcloudy/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,freedomtan/tensorflow,tensorflow/tensorflow,gunan/tensorflow,asimshankar/tensorflow,dancingdan/tensorflow,dendisuhubdy/tensorflow,jhseu/tensorflow,jalexvig/tensorflow,cxxgtxy/tensorflow,yongtang/tensorflow,gunan/tensorflow,xodus7/tensorflow,kevin-coder/tensorflow-fork,dancingdan/tensorflow,sarvex/tensorflow,gojira/tensorflow,frreiss/tensorflow-fred,arborh/tensorflow,tensorflow/tensorflow,chemelnucfin/tensorflow,renyi533/tensorflow,snnn/tensorflow,alshedivat/tensorflow,alsrgv/tensorflow,seanli9jan/tensorflow,AnishShah/tensorflow,yongtang/tensorflow,jbedorf/tensorflow,frreiss/tensorflow-fred,brchiu/tensorflow,adit-chandra/tensorflow,jbedorf/tensorflow,drpngx/tensorflow,apark263/tensorflow,AnishShah/tensorflow,alsrgv/tensorflow,paolodedios/tensorflow,alsrgv/tensorflow,brchiu/tensorflow,dongjoon-hyun/tensorflow,lukeiwanski/tensorflow,freedomtan/tensorflow,aldian/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,girving/tensorflow,yanchen036/tensorflow,dongjoon-hyun/tensorflow,Intel-tensorflow/tensorflow,tensorflow/tensorflow-pywrap_saved_model,tensorflow/tensorflow-pywrap_tf_optimizer,aselle/tensorflow,girving/tensorflow,DavidNorman/tensorflow,DavidNorman/tensorflow,hfp/tensorflow-xsmm,lukeiwanski/tensorflow,brchiu/tensorflow,aselle/tensorflow,meteorcloudy/tensorflow,Intel-Corporation/tensorflow,aselle/tensorflow,petewarden/tensorflow,adit-chandra/tensorflow,alsrgv/tensorflow,AnishShah/tensorflow,dongjoon-hyun/tensorflow,aam-at/tensorflow,chemelnucfin/tensorflow,xzturn/tensorflow,alsrgv/tensorflow,davidzchen/tensorflow,ppwwyyxx/tensorflow,ageron/tensorflow,yanchen036/tensorflow,jart/tensorflow,aam-at/tensorflow,paolodedios/tensorflow,caisq/tensorflow,petewarden/tensorflow,caisq/tensorflow,ZhangXinNan/tensorflow,jhseu/tensorflow,DavidNorman/tensorflow,arborh/tensorflow,tensorflow/tensorflow-experimental_link_static_libraries_once,lukeiwanski/tensorflow,tensorflow/tensorflow-pywrap_tf_optimizer,aam-at/tensorflow,aldian/tensorflow,karllessard/tensorflow,seanli9jan/tensorflow,ghchinoy/tensorflow,paolodedios/tensorflow,seanli9jan/tensorflow,caisq/tensorflow,tensorflow/tensorflow,tensorflow/tensorflow-pywrap_saved_model,dendisuhubdy/tensorflow,seanli9jan/tensorflow,ghchinoy/tensorflow,dongjoon-hyun/tensorflow,tensorflow/tensorflow,jalexvig/tensorflow,gunan/tensorflow,ghchinoy/tensorflow,Intel-tensorflow/tensorflow,tensorflow/tensorflow