Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Show, Control and Tell: A Framework for Generating Controllable and
  Grounded Captions   Current captioning approaches can describe images using black-box
architectures whose behavior is hardly controllable and explainable from the
exterior. As an image can be described in infinite ways depending on the goal
and the context at hand, a higher degree of controllability is needed to apply
captioning algorithms in complex scenarios. In this paper, we introduce a novel
framework for image captioning which can generate diverse descriptions by
allowing both grounding and controllability. Given a control signal in the form
of a sequence or set of image regions, we generate the corresponding caption
through a recurrent architecture which predicts textual chunks explicitly
grounded on regions, following the constraints of the given control.
Experiments are conducted on Flickr30k Entities and on COCO Entities, an
extended version of COCO in which we add grounding annotations collected in a
semi-automatic manner. Results demonstrate that our method achieves state of
the art performances on controllable image captioning, in terms of caption
quality and diversity. Code and annotations are publicly available at:
https://github.com/aimagelab/show-control-and-tell.

17
The categories of this paper are: q-fin
14
The categories of this paper are: math-ph
7
The categories of this paper are: hep-th
13
The categories of this paper are: quant-ph
35
The categories of this paper are: mtrl-th
21
The categories of this paper are: adap-org
32
The categories of this paper are: atom-ph
6
The categories of this paper are: astro-ph
3
The categories of this paper are: physics
29
The categories of this paper are: funct-an
34
The categories of this paper are: chem-ph
26
The categories of this paper are: bayes-an
23
The categories of this paper are: solv-int
12
The categories of this paper are: nucl-th
15
The categories of this paper are: nucl-ex
22
The categories of this paper are: chao-dyn