Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Telling the What while Pointing to the Where: Multimodal Queries for
  Image Retrieval   Most existing image retrieval systems use text queries as a way for the user
to express what they are looking for. However, fine-grained image retrieval
often requires the ability to also express where in the image the content they
are looking for is. The text modality can only cumbersomely express such
localization preferences, whereas pointing is a more natural fit. In this
paper, we propose an image retrieval setup with a new form of multimodal
queries, where the user simultaneously uses both spoken natural language (the
what) and mouse traces over an empty canvas (the where) to express the
characteristics of the desired target image. We then describe simple
modifications to an existing image retrieval model, enabling it to operate in
this setup. Qualitative and quantitative experiments show that our model
effectively takes this spatial guidance into account, and provides
significantly more accurate retrieval results compared to text-only equivalent
systems.

19
The categories of this paper are: eess
8
The categories of this paper are: hep-ex
18
The categories of this paper are: econ
6
The categories of this paper are: astro-ph
27
The categories of this paper are: comp-gas
2
The categories of this paper are: math
34
The categories of this paper are: chem-ph
15
The categories of this paper are: nucl-ex
37
The categories of this paper are: supr-con
9
The categories of this paper are: nlin
10
The categories of this paper are: q-bio
5
The categories of this paper are: gr-qc
0
The categories of this paper are: hep-ph
25
The categories of this paper are: patt-sol
35
The categories of this paper are: mtrl-th
26
The categories of this paper are: bayes-an