Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: An Exponential Improvement on the Memorization Capacity of Deep
  Threshold Networks   It is well known that modern deep neural networks are powerful enough to
memorize datasets even when the labels have been randomized. Recently,
Vershynin (2020) settled a long standing question by Baum (1988), proving that
\emph{deep threshold} networks can memorize $n$ points in $d$ dimensions using
$\widetilde{\mathcal{O}}(e^{1/\delta^2}+\sqrt{n})$ neurons and
$\widetilde{\mathcal{O}}(e^{1/\delta^2}(d+\sqrt{n})+n)$ weights, where $\delta$
is the minimum distance between the points. In this work, we improve the
dependence on $\delta$ from exponential to almost linear, proving that
$\widetilde{\mathcal{O}}(\frac{1}{\delta}+\sqrt{n})$ neurons and
$\widetilde{\mathcal{O}}(\frac{d}{\delta}+n)$ weights are sufficient. Our
construction uses Gaussian random weights only in the first layer, while all
the subsequent layers use binary or integer weights. We also prove new lower
bounds by connecting memorization in neural networks to the purely geometric
problem of separating $n$ points on a sphere using hyperplanes.

32
The categories of this paper are: atom-ph
29
The categories of this paper are: funct-an
28
The categories of this paper are: alg-geom
27
The categories of this paper are: comp-gas
15
The categories of this paper are: nucl-ex
22
The categories of this paper are: chao-dyn
35
The categories of this paper are: mtrl-th
13
The categories of this paper are: quant-ph
26
The categories of this paper are: bayes-an
4
The categories of this paper are: cond-mat
20
The categories of this paper are: acc-phys
37
The categories of this paper are: supr-con
9
The categories of this paper are: nlin
33
The categories of this paper are: plasm-ph
3
The categories of this paper are: physics
12
The categories of this paper are: nucl-th