Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Neuron with Steady Response Leads to Better Generalization   Regularization can mitigate the generalization gap between training and
inference by introducing inductive bias. Existing works have already proposed
various inductive biases from diverse perspectives. However, to the best of our
knowledge, none of them explores inductive bias from the perspective of
class-dependent response distribution of individual neurons. In this paper, we
conduct a substantial analysis of the characteristics of such distribution.
Based on the analysis results, we articulate the Neuron Steadiness Hypothesis:
the neuron with similar responses to instances of the same class leads to
better generalization. Accordingly, we propose a new regularization method
called Neuron Steadiness Regularization to reduce neuron intra-class response
variance. We conduct extensive experiments on Multilayer Perceptron,
Convolutional Neural Network, and Graph Neural Network with popular benchmark
datasets of diverse domains, which show that our Neuron Steadiness
Regularization consistently outperforms the vanilla version of models with
significant gain and low additional overhead.

2
The categories of this paper are: math
30
The categories of this paper are: q-alg
18
The categories of this paper are: econ
3
The categories of this paper are: physics
12
The categories of this paper are: nucl-th
35
The categories of this paper are: mtrl-th
21
The categories of this paper are: adap-org
36
The categories of this paper are: cmp-lg
34
The categories of this paper are: chem-ph
9
The categories of this paper are: nlin
27
The categories of this paper are: comp-gas
11
The categories of this paper are: hep-lat
22
The categories of this paper are: chao-dyn
25
The categories of this paper are: patt-sol
15
The categories of this paper are: nucl-ex
16
The categories of this paper are: stat