Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Hidden-Fold Networks: Random Recurrent Residuals Using Sparse Supermasks   Deep neural networks (DNNs) are so over-parametrized that recent research has
found them to already contain a subnetwork with high accuracy at their randomly
initialized state. Finding these subnetworks is a viable alternative training
method to weight learning. In parallel, another line of work has hypothesized
that deep residual networks (ResNets) are trying to approximate the behaviour
of shallow recurrent neural networks (RNNs) and has proposed a way for
compressing them into recurrent models. This paper proposes blending these
lines of research into a highly compressed yet accurate model: Hidden-Fold
Networks (HFNs). By first folding ResNet into a recurrent structure and then
searching for an accurate subnetwork hidden within the randomly initialized
model, a high-performing yet tiny HFN is obtained without ever updating the
weights. As a result, HFN achieves equivalent performance to ResNet50 on
CIFAR100 while occupying 38.5x less memory, and similar performance to ResNet34
on ImageNet with a memory size 26.8x smaller. The HFN will become even more
attractive by minimizing data transfers while staying accurate when it runs on
highly-quantized and randomly-weighted DNN inference accelerators. Code
available at https://github.com/Lopez-Angel/hidden-fold-networks

26
The categories of this paper are: bayes-an
2
The categories of this paper are: math
16
The categories of this paper are: stat
21
The categories of this paper are: adap-org
18
The categories of this paper are: econ
24
The categories of this paper are: dg-ga
34
The categories of this paper are: chem-ph
32
The categories of this paper are: atom-ph
28
The categories of this paper are: alg-geom
4
The categories of this paper are: cond-mat
33
The categories of this paper are: plasm-ph
9
The categories of this paper are: nlin
37
The categories of this paper are: supr-con
0
The categories of this paper are: hep-ph
3
The categories of this paper are: physics
20
The categories of this paper are: acc-phys