Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image
  Captioning   Attention-based neural encoder-decoder frameworks have been widely adopted
for image captioning. Most methods force visual attention to be active for
every generated word. However, the decoder likely requires little to no visual
information from the image to predict non-visual words such as "the" and "of".
Other words that may seem visual can often be predicted reliably just from the
language model e.g., "sign" after "behind a red stop" or "phone" following
"talking on a cell". In this paper, we propose a novel adaptive attention model
with a visual sentinel. At each time step, our model decides whether to attend
to the image (and if so, to which regions) or to the visual sentinel. The model
decides whether to attend to the image and where, in order to extract
meaningful information for sequential word generation. We test our method on
the COCO image captioning 2015 challenge dataset and Flickr30K. Our approach
sets the new state-of-the-art by a significant margin.

19
The categories of this paper are: eess
35
The categories of this paper are: mtrl-th
24
The categories of this paper are: dg-ga
22
The categories of this paper are: chao-dyn
3
The categories of this paper are: physics
29
The categories of this paper are: funct-an
26
The categories of this paper are: bayes-an
18
The categories of this paper are: econ
15
The categories of this paper are: nucl-ex
9
The categories of this paper are: nlin
0
The categories of this paper are: hep-ph
7
The categories of this paper are: hep-th
12
The categories of this paper are: nucl-th
8
The categories of this paper are: hep-ex
32
The categories of this paper are: atom-ph
10
The categories of this paper are: q-bio