Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Multimodal Integration of Human-Like Attention in Visual Question
  Answering   Human-like attention as a supervisory signal to guide neural attention has
shown significant promise but is currently limited to uni-modal integration -
even for inherently multimodal tasks such as visual question answering (VQA).
We present the Multimodal Human-like Attention Network (MULAN) - the first
method for multimodal integration of human-like attention on image and text
during training of VQA models. MULAN integrates attention predictions from two
state-of-the-art text and image saliency models into neural self-attention
layers of a recent transformer-based VQA model. Through evaluations on the
challenging VQAv2 dataset, we show that MULAN achieves a new state-of-the-art
performance of 73.98% accuracy on test-std and 73.72% on test-dev and, at the
same time, has approximately 80% fewer trainable parameters than prior work.
Overall, our work underlines the potential of integrating multimodal human-like
and neural attention for VQA

21
The categories of this paper are: adap-org
18
The categories of this paper are: econ
20
The categories of this paper are: acc-phys
12
The categories of this paper are: nucl-th
31
The categories of this paper are: ao-sci
16
The categories of this paper are: stat
33
The categories of this paper are: plasm-ph
10
The categories of this paper are: q-bio
7
The categories of this paper are: hep-th
19
The categories of this paper are: eess
24
The categories of this paper are: dg-ga
5
The categories of this paper are: gr-qc
29
The categories of this paper are: funct-an
30
The categories of this paper are: q-alg
34
The categories of this paper are: chem-ph
0
The categories of this paper are: hep-ph