Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Go Small and Similar: A Simple Output Decay Brings Better Performance   Regularization and data augmentation methods have been widely used and become
increasingly indispensable in deep learning training. Researchers who devote
themselves to this have considered various possibilities. But so far, there has
been little discussion about regularizing outputs of the model. This paper
begins with empirical observations that better performances are significantly
associated with output distributions, that have smaller average values and
variances. By audaciously assuming there is causality involved, we propose a
novel regularization term, called Output Decay, that enforces the model to
assign smaller and similar output values on each class. Though being
counter-intuitive, such a small modification result in a remarkable improvement
on performance. Extensive experiments demonstrate the wide applicability,
versatility, and compatibility of Output Decay.

33
The categories of this paper are: plasm-ph
34
The categories of this paper are: chem-ph
19
The categories of this paper are: eess
28
The categories of this paper are: alg-geom
7
The categories of this paper are: hep-th
8
The categories of this paper are: hep-ex
35
The categories of this paper are: mtrl-th
9
The categories of this paper are: nlin
36
The categories of this paper are: cmp-lg
11
The categories of this paper are: hep-lat
20
The categories of this paper are: acc-phys
24
The categories of this paper are: dg-ga
32
The categories of this paper are: atom-ph
23
The categories of this paper are: solv-int
12
The categories of this paper are: nucl-th
10
The categories of this paper are: q-bio