Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Can audio-visual integration strengthen robustness under multimodal
  attacks?   In this paper, we propose to make a systematic study on machines multisensory
perception under attacks. We use the audio-visual event recognition task
against multimodal adversarial attacks as a proxy to investigate the robustness
of audio-visual learning. We attack audio, visual, and both modalities to
explore whether audio-visual integration still strengthens perception and how
different fusion mechanisms affect the robustness of audio-visual models. For
interpreting the multimodal interactions under attacks, we learn a
weakly-supervised sound source visual localization model to localize sounding
regions in videos. To mitigate multimodal attacks, we propose an audio-visual
defense approach based on an audio-visual dissimilarity constraint and external
feature memory banks. Extensive experiments demonstrate that audio-visual
models are susceptible to multimodal adversarial attacks; audio-visual
integration could decrease the model robustness rather than strengthen under
multimodal attacks; even a weakly-supervised sound source visual localization
model can be successfully fooled; our defense method can improve the
invulnerability of audio-visual networks without significantly sacrificing
clean model performance.

29
The categories of this paper are: funct-an
32
The categories of this paper are: atom-ph
14
The categories of this paper are: math-ph
18
The categories of this paper are: econ
10
The categories of this paper are: q-bio
7
The categories of this paper are: hep-th
11
The categories of this paper are: hep-lat
12
The categories of this paper are: nucl-th
36
The categories of this paper are: cmp-lg
8
The categories of this paper are: hep-ex
23
The categories of this paper are: solv-int
22
The categories of this paper are: chao-dyn
27
The categories of this paper are: comp-gas
2
The categories of this paper are: math
4
The categories of this paper are: cond-mat
6
The categories of this paper are: astro-ph