Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Adding Interpretable Attention to Neural Translation Models Improves
  Word Alignment   Multi-layer models with multiple attention heads per layer provide superior
translation quality compared to simpler and shallower models, but determining
what source context is most relevant to each target word is more challenging as
a result. Therefore, deriving high-accuracy word alignments from the
activations of a state-of-the-art neural machine translation model is an open
challenge. We propose a simple model extension to the Transformer architecture
that makes use of its hidden representations and is restricted to attend solely
on encoder information to predict the next word. It can be trained on bilingual
data without word-alignment information. We further introduce a novel alignment
inference procedure which applies stochastic gradient descent to directly
optimize the attention activations towards a given target word. The resulting
alignments dramatically outperform the naive approach to interpreting
Transformer attention activations, and are comparable to Giza++ on two publicly
available data sets.

35
The categories of this paper are: mtrl-th
36
The categories of this paper are: cmp-lg
13
The categories of this paper are: quant-ph
25
The categories of this paper are: patt-sol
0
The categories of this paper are: hep-ph
14
The categories of this paper are: math-ph
10
The categories of this paper are: q-bio
2
The categories of this paper are: math
6
The categories of this paper are: astro-ph
15
The categories of this paper are: nucl-ex
33
The categories of this paper are: plasm-ph
8
The categories of this paper are: hep-ex
32
The categories of this paper are: atom-ph
7
The categories of this paper are: hep-th
21
The categories of this paper are: adap-org
27
The categories of this paper are: comp-gas