Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Learn To Remember: Transformer with Recurrent Memory for Document-Level
  Machine Translation   The Transformer architecture has led to significant gains in machine
translation. However, most studies focus on only sentence-level translation
without considering the context dependency within documents, leading to the
inadequacy of document-level coherence. Some recent research tried to mitigate
this issue by introducing an additional context encoder or translating with
multiple sentences or even the entire document. Such methods may lose the
information on the target side or have an increasing computational complexity
as documents get longer. To address such problems, we introduce a recurrent
memory unit to the vanilla Transformer, which supports the information exchange
between the sentence and previous context. The memory unit is recurrently
updated by acquiring information from sentences, and passing the aggregated
knowledge back to subsequent sentence states. We follow a two-stage training
strategy, in which the model is first trained at the sentence level and then
finetuned for document-level translation. We conduct experiments on three
popular datasets for document-level machine translation and our model has an
average improvement of 0.91 s-BLEU over the sentence-level baseline. We also
achieve state-of-the-art results on TED and News, outperforming the previous
work by 0.36 s-BLEU and 1.49 d-BLEU on average.

37
The categories of this paper are: supr-con
26
The categories of this paper are: bayes-an
4
The categories of this paper are: cond-mat
2
The categories of this paper are: math
13
The categories of this paper are: quant-ph
33
The categories of this paper are: plasm-ph
6
The categories of this paper are: astro-ph
10
The categories of this paper are: q-bio
20
The categories of this paper are: acc-phys
32
The categories of this paper are: atom-ph
30
The categories of this paper are: q-alg
5
The categories of this paper are: gr-qc
25
The categories of this paper are: patt-sol
22
The categories of this paper are: chao-dyn
31
The categories of this paper are: ao-sci
35
The categories of this paper are: mtrl-th