Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Learning and Generalization in Overparameterized Neural Networks, Going
  Beyond Two Layers   The fundamental learning theory behind neural networks remains largely open.
What classes of functions can neural networks actually learn? Why doesn't the
trained network overfit when it is overparameterized?
  In this work, we prove that overparameterized neural networks can learn some
notable concept classes, including two and three-layer networks with fewer
parameters and smooth activations. Moreover, the learning can be simply done by
SGD (stochastic gradient descent) or its variants in polynomial time using
polynomially many samples. The sample complexity can also be almost independent
of the number of parameters in the network.
  On the technique side, our analysis goes beyond the so-called NTK (neural
tangent kernel) linearization of neural networks in prior works. We establish a
new notion of quadratic approximation of the neural network (that can be viewed
as a second-order variant of NTK), and connect it to the SGD theory of escaping
saddle points.

11
The categories of this paper are: hep-lat
9
The categories of this paper are: nlin
0
The categories of this paper are: hep-ph
22
The categories of this paper are: chao-dyn
14
The categories of this paper are: math-ph
24
The categories of this paper are: dg-ga
26
The categories of this paper are: bayes-an
7
The categories of this paper are: hep-th
29
The categories of this paper are: funct-an
35
The categories of this paper are: mtrl-th
33
The categories of this paper are: plasm-ph
36
The categories of this paper are: cmp-lg
30
The categories of this paper are: q-alg
3
The categories of this paper are: physics
31
The categories of this paper are: ao-sci
17
The categories of this paper are: q-fin