Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Limitation of characterizing implicit regularization by data-independent
  functions   In recent years, understanding the implicit regularization of neural networks
(NNs) has become a central task of deep learning theory. However, implicit
regularization is in itself not completely defined and well understood. In this
work, we make an attempt to mathematically define and study the implicit
regularization. Importantly, we explore the limitation of a common approach of
characterizing the implicit regularization by data-independent functions. We
propose two dynamical mechanisms, i.e., Two-point and One-point Overlapping
mechanisms, based on which we provide two recipes for producing classes of
one-hidden-neuron NNs that provably cannot be fully characterized by a type of
or all data-independent functions. Our results signify the profound
data-dependency of implicit regularization in general, inspiring us to study in
detail the data-dependency of NN implicit regularization in the future.

33
The categories of this paper are: plasm-ph
7
The categories of this paper are: hep-th
35
The categories of this paper are: mtrl-th
0
The categories of this paper are: hep-ph
20
The categories of this paper are: acc-phys
23
The categories of this paper are: solv-int
2
The categories of this paper are: math
3
The categories of this paper are: physics
29
The categories of this paper are: funct-an
16
The categories of this paper are: stat
6
The categories of this paper are: astro-ph
13
The categories of this paper are: quant-ph
14
The categories of this paper are: math-ph
21
The categories of this paper are: adap-org
24
The categories of this paper are: dg-ga
19
The categories of this paper are: eess