Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Spectra of the Conjugate Kernel and Neural Tangent Kernel for
  linear-width neural networks   We study the eigenvalue distributions of the Conjugate Kernel and Neural
Tangent Kernel associated to multi-layer feedforward neural networks. In an
asymptotic regime where network width is increasing linearly in sample size,
under random initialization of the weights, and for input samples satisfying a
notion of approximate pairwise orthogonality, we show that the eigenvalue
distributions of the CK and NTK converge to deterministic limits. The limit for
the CK is described by iterating the Marcenko-Pastur map across the hidden
layers. The limit for the NTK is equivalent to that of a linear combination of
the CK matrices across layers, and may be described by recursive fixed-point
equations that extend this Marcenko-Pastur map. We demonstrate the agreement of
these asymptotic predictions with the observed spectra for both synthetic and
CIFAR-10 training data, and we perform a small simulation to investigate the
evolutions of these spectra over training.

26
The categories of this paper are: bayes-an
33
The categories of this paper are: plasm-ph
31
The categories of this paper are: ao-sci
11
The categories of this paper are: hep-lat
15
The categories of this paper are: nucl-ex
22
The categories of this paper are: chao-dyn
23
The categories of this paper are: solv-int
35
The categories of this paper are: mtrl-th
13
The categories of this paper are: quant-ph
32
The categories of this paper are: atom-ph
9
The categories of this paper are: nlin
6
The categories of this paper are: astro-ph
30
The categories of this paper are: q-alg
29
The categories of this paper are: funct-an
14
The categories of this paper are: math-ph
27
The categories of this paper are: comp-gas