Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: CapWAP: Captioning with a Purpose   The traditional image captioning task uses generic reference captions to
provide textual information about images. Different user populations, however,
will care about different visual aspects of images. In this paper, we propose a
new task, Captioning with a Purpose (CapWAP). Our goal is to develop systems
that can be tailored to be useful for the information needs of an intended
population, rather than merely provide generic information about an image. In
this task, we use question-answer (QA) pairs---a natural expression of
information need---from users, instead of reference captions, for both training
and post-inference evaluation. We show that it is possible to use reinforcement
learning to directly optimize for the intended information need, by rewarding
outputs that allow a question answering model to provide correct answers to
sampled user questions. We convert several visual question answering datasets
into CapWAP datasets, and demonstrate that under a variety of scenarios our
purposeful captioning system learns to anticipate and fulfill specific
information needs better than its generic counterparts, as measured by QA
performance on user questions from unseen images, when using the caption alone
as context.

5
The categories of this paper are: gr-qc
31
The categories of this paper are: ao-sci
20
The categories of this paper are: acc-phys
30
The categories of this paper are: q-alg
23
The categories of this paper are: solv-int
19
The categories of this paper are: eess
4
The categories of this paper are: cond-mat
37
The categories of this paper are: supr-con
25
The categories of this paper are: patt-sol
29
The categories of this paper are: funct-an
15
The categories of this paper are: nucl-ex
17
The categories of this paper are: q-fin
26
The categories of this paper are: bayes-an
24
The categories of this paper are: dg-ga
21
The categories of this paper are: adap-org
35
The categories of this paper are: mtrl-th