Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Encoding-based Memory Modules for Recurrent Neural Networks   Learning to solve sequential tasks with recurrent models requires the ability
to memorize long sequences and to extract task-relevant features from them. In
this paper, we study the memorization subtask from the point of view of the
design and training of recurrent neural networks. We propose a new model, the
Linear Memory Network, which features an encoding-based memorization component
built with a linear autoencoder for sequences. We extend the memorization
component with a modular memory that encodes the hidden state sequence at
different sampling frequencies. Additionally, we provide a specialized training
algorithm that initializes the memory to efficiently encode the hidden
activations of the network. The experimental results on synthetic and
real-world datasets show that specializing the training algorithm to train the
memorization component always improves the final performance whenever the
memorization of long sequences is necessary to solve the problem.

30
The categories of this paper are: q-alg
2
The categories of this paper are: math
37
The categories of this paper are: supr-con
28
The categories of this paper are: alg-geom
24
The categories of this paper are: dg-ga
5
The categories of this paper are: gr-qc
11
The categories of this paper are: hep-lat
21
The categories of this paper are: adap-org
29
The categories of this paper are: funct-an
20
The categories of this paper are: acc-phys
32
The categories of this paper are: atom-ph
23
The categories of this paper are: solv-int
26
The categories of this paper are: bayes-an
35
The categories of this paper are: mtrl-th
14
The categories of this paper are: math-ph
18
The categories of this paper are: econ