Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Regularizing by the Variance of the Activations' Sample-Variances   Normalization techniques play an important role in supporting efficient and
often more effective training of deep neural networks. While conventional
methods explicitly normalize the activations, we suggest to add a loss term
instead. This new loss term encourages the variance of the activations to be
stable and not vary from one random mini-batch to the next. As we prove, this
encourages the activations to be distributed around a few distinct modes. We
also show that if the inputs are from a mixture of two Gaussians, the new loss
would either join the two together, or separate between them optimally in the
LDA sense, depending on the prior probabilities. Finally, we are able to link
the new regularization term to the batchnorm method, which provides it with a
regularization perspective. Our experiments demonstrate an improvement in
accuracy over the batchnorm technique for both CNNs and fully connected
networks.

12
The categories of this paper are: nucl-th
5
The categories of this paper are: gr-qc
22
The categories of this paper are: chao-dyn
26
The categories of this paper are: bayes-an
18
The categories of this paper are: econ
34
The categories of this paper are: chem-ph
14
The categories of this paper are: math-ph
37
The categories of this paper are: supr-con
36
The categories of this paper are: cmp-lg
33
The categories of this paper are: plasm-ph
3
The categories of this paper are: physics
25
The categories of this paper are: patt-sol
31
The categories of this paper are: ao-sci
6
The categories of this paper are: astro-ph
13
The categories of this paper are: quant-ph
4
The categories of this paper are: cond-mat