Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Token Drop mechanism for Neural Machine Translation   Neural machine translation with millions of parameters is vulnerable to
unfamiliar inputs. We propose Token Drop to improve generalization and avoid
overfitting for the NMT model. Similar to word dropout, whereas we replace
dropped token with a special token instead of setting zero to words. We further
introduce two self-supervised objectives: Replaced Token Detection and Dropped
Token Prediction. Our method aims to force model generating target translation
with less information, in this way the model can learn textual representation
better. Experiments on Chinese-English and English-Romanian benchmark
demonstrate the effectiveness of our approach and our model achieves
significant improvements over a strong Transformer baseline.

36
The categories of this paper are: cmp-lg
2
The categories of this paper are: math
23
The categories of this paper are: solv-int
12
The categories of this paper are: nucl-th
26
The categories of this paper are: bayes-an
17
The categories of this paper are: q-fin
0
The categories of this paper are: hep-ph
10
The categories of this paper are: q-bio
20
The categories of this paper are: acc-phys
16
The categories of this paper are: stat
24
The categories of this paper are: dg-ga
5
The categories of this paper are: gr-qc
14
The categories of this paper are: math-ph
7
The categories of this paper are: hep-th
33
The categories of this paper are: plasm-ph
34
The categories of this paper are: chem-ph