Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Pre-training Multilingual Neural Machine Translation by Leveraging
  Alignment Information   We investigate the following question for machine translation (MT): can we
develop a single universal MT model to serve as the common seed and obtain
derivative and improved models on arbitrary language pairs? We propose mRASP,
an approach to pre-train a universal multilingual neural machine translation
model. Our key idea in mRASP is its novel technique of random aligned
substitution, which brings words and phrases with similar meanings across
multiple languages closer in the representation space. We pre-train a mRASP
model on 32 language pairs jointly with only public datasets. The model is then
fine-tuned on downstream language pairs to obtain specialized MT models. We
carry out extensive experiments on 42 translation directions across a diverse
settings, including low, medium, rich resource, and as well as transferring to
exotic language pairs. Experimental results demonstrate that mRASP achieves
significant performance improvement compared to directly training on those
target pairs. It is the first time to verify that multiple low-resource
language pairs can be utilized to improve rich resource MT. Surprisingly, mRASP
is even able to improve the translation quality on exotic languages that never
occur in the pre-training corpus. Code, data, and pre-trained models are
available at https://github.com/linzehui/mRASP.

22
The categories of this paper are: chao-dyn
36
The categories of this paper are: cmp-lg
12
The categories of this paper are: nucl-th
29
The categories of this paper are: funct-an
6
The categories of this paper are: astro-ph
19
The categories of this paper are: eess
2
The categories of this paper are: math
30
The categories of this paper are: q-alg
28
The categories of this paper are: alg-geom
31
The categories of this paper are: ao-sci
16
The categories of this paper are: stat
0
The categories of this paper are: hep-ph
24
The categories of this paper are: dg-ga
10
The categories of this paper are: q-bio
18
The categories of this paper are: econ
11
The categories of this paper are: hep-lat