Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Low-rank passthrough neural networks   Various common deep learning architectures, such as LSTMs, GRUs, Resnets and
Highway Networks, employ state passthrough connections that support training
with high feed-forward depth or recurrence over many time steps. These
"Passthrough Networks" architectures also enable the decoupling of the network
state size from the number of parameters of the network, a possibility has been
studied by \newcite{Sak2014} with their low-rank parametrization of the LSTM.
In this work we extend this line of research, proposing effective, low-rank and
low-rank plus diagonal matrix parametrizations for Passthrough Networks which
exploit this decoupling property, reducing the data complexity and memory
requirements of the network while preserving its memory capacity. This is
particularly beneficial in low-resource settings as it supports expressive
models with a compact parametrization less susceptible to overfitting. We
present competitive experimental results on several tasks, including language
modeling and a near state of the art result on sequential randomly-permuted
MNIST classification, a hard task on natural data.

10
The categories of this paper are: q-bio
32
The categories of this paper are: atom-ph
21
The categories of this paper are: adap-org
22
The categories of this paper are: chao-dyn
18
The categories of this paper are: econ
35
The categories of this paper are: mtrl-th
17
The categories of this paper are: q-fin
24
The categories of this paper are: dg-ga
0
The categories of this paper are: hep-ph
14
The categories of this paper are: math-ph
5
The categories of this paper are: gr-qc
31
The categories of this paper are: ao-sci
12
The categories of this paper are: nucl-th
25
The categories of this paper are: patt-sol
20
The categories of this paper are: acc-phys
34
The categories of this paper are: chem-ph