Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: On the exact computation of linear frequency principle dynamics and its
  generalization   Recent works show an intriguing phenomenon of Frequency Principle
(F-Principle) that deep neural networks (DNNs) fit the target function from low
to high frequency during the training, which provides insight into the training
and generalization behavior of DNNs in complex tasks. In this paper, through
analysis of an infinite-width two-layer NN in the neural tangent kernel (NTK)
regime, we derive the exact differential equation, namely Linear
Frequency-Principle (LFP) model, governing the evolution of NN output function
in the frequency domain during the training. Our exact computation applies for
general activation functions with no assumption on size and distribution of
training data. This LFP model unravels that higher frequencies evolve
polynomially or exponentially slower than lower frequencies depending on the
smoothness/regularity of the activation function. We further bridge the gap
between training dynamics and generalization by proving that LFP model
implicitly minimizes a Frequency-Principle norm (FP-norm) of the learned
function, by which higher frequencies are more severely penalized depending on
the inverse of their evolution rate. Finally, we derive an \textit{a priori}
generalization error bound controlled by the FP-norm of the target function,
which provides a theoretical justification for the empirical results that DNNs
often generalize well for low frequency functions.

20
The categories of this paper are: acc-phys
28
The categories of this paper are: alg-geom
6
The categories of this paper are: astro-ph
9
The categories of this paper are: nlin
32
The categories of this paper are: atom-ph
29
The categories of this paper are: funct-an
35
The categories of this paper are: mtrl-th
16
The categories of this paper are: stat
4
The categories of this paper are: cond-mat
31
The categories of this paper are: ao-sci
37
The categories of this paper are: supr-con
17
The categories of this paper are: q-fin
34
The categories of this paper are: chem-ph
2
The categories of this paper are: math
14
The categories of this paper are: math-ph
12
The categories of this paper are: nucl-th