Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: A Neural Scaling Law from the Dimension of the Data Manifold   When data is plentiful, the loss achieved by well-trained neural networks
scales as a power-law $L \propto N^{-\alpha}$ in the number of network
parameters $N$. This empirical scaling law holds for a wide variety of data
modalities, and may persist over many orders of magnitude. The scaling law can
be explained if neural models are effectively just performing regression on a
data manifold of intrinsic dimension $d$. This simple theory predicts that the
scaling exponents $\alpha \approx 4/d$ for cross-entropy and mean-squared error
losses. We confirm the theory by independently measuring the intrinsic
dimension and the scaling exponents in a teacher/student framework, where we
can study a variety of $d$ and $\alpha$ by dialing the properties of random
teacher networks. We also test the theory with CNN image classifiers on several
datasets and with GPT-type language models.

6
The categories of this paper are: astro-ph
9
The categories of this paper are: nlin
10
The categories of this paper are: q-bio
19
The categories of this paper are: eess
8
The categories of this paper are: hep-ex
34
The categories of this paper are: chem-ph
15
The categories of this paper are: nucl-ex
30
The categories of this paper are: q-alg
23
The categories of this paper are: solv-int
17
The categories of this paper are: q-fin
32
The categories of this paper are: atom-ph
37
The categories of this paper are: supr-con
36
The categories of this paper are: cmp-lg
26
The categories of this paper are: bayes-an
7
The categories of this paper are: hep-th
2
The categories of this paper are: math