Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Audio-visual speech separation based on joint feature representation
  with cross-modal attention   Multi-modal based speech separation has exhibited a specific advantage on
isolating the target character in multi-talker noisy environments.
Unfortunately, most of current separation strategies prefer a straightforward
fusion based on feature learning of each single modality, which is far from
sufficient consideration of inter-relationships between modalites. Inspired by
learning joint feature representations from audio and visual streams with
attention mechanism, in this study, a novel cross-modal fusion strategy is
proposed to benefit the whole framework with semantic correlations between
different modalities. To further improve audio-visual speech separation, the
dense optical flow of lip motion is incorporated to strengthen the robustness
of visual representation. The evaluation of the proposed work is performed on
two public audio-visual speech separation benchmark datasets. The overall
improvement of the performance has demonstrated that the additional motion
network effectively enhances the visual representation of the combined lip
images and audio signal, as well as outperforming the baseline in terms of all
metrics with the proposed cross-modal fusion.

32
The categories of this paper are: atom-ph
37
The categories of this paper are: supr-con
0
The categories of this paper are: hep-ph
22
The categories of this paper are: chao-dyn
21
The categories of this paper are: adap-org
24
The categories of this paper are: dg-ga
12
The categories of this paper are: nucl-th
30
The categories of this paper are: q-alg
14
The categories of this paper are: math-ph
27
The categories of this paper are: comp-gas
34
The categories of this paper are: chem-ph
17
The categories of this paper are: q-fin
20
The categories of this paper are: acc-phys
18
The categories of this paper are: econ
9
The categories of this paper are: nlin
5
The categories of this paper are: gr-qc