Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Cross-Modal Generalization: Learning in Low Resource Modalities via
  Meta-Alignment   The natural world is abundant with concepts expressed via visual, acoustic,
tactile, and linguistic modalities. Much of the existing progress in multimodal
learning, however, focuses primarily on problems where the same set of
modalities are present at train and test time, which makes learning in
low-resource modalities particularly difficult. In this work, we propose
algorithms for cross-modal generalization: a learning paradigm to train a model
that can (1) quickly perform new tasks in a target modality (i.e.
meta-learning) and (2) doing so while being trained on a different source
modality. We study a key research question: how can we ensure generalization
across modalities despite using separate encoders for different source and
target modalities? Our solution is based on meta-alignment, a novel method to
align representation spaces using strongly and weakly paired cross-modal data
while ensuring quick generalization to new tasks across different modalities.
We study this problem on 3 classification tasks: text to image, image to audio,
and text to speech. Our results demonstrate strong performance even when the
new target modality has only a few (1-10) labeled samples and in the presence
of noisy labels, a scenario particularly prevalent in low-resource modalities.

2
The categories of this paper are: math
20
The categories of this paper are: acc-phys
25
The categories of this paper are: patt-sol
4
The categories of this paper are: cond-mat
11
The categories of this paper are: hep-lat
36
The categories of this paper are: cmp-lg
27
The categories of this paper are: comp-gas
16
The categories of this paper are: stat
22
The categories of this paper are: chao-dyn
31
The categories of this paper are: ao-sci
33
The categories of this paper are: plasm-ph
28
The categories of this paper are: alg-geom
26
The categories of this paper are: bayes-an
10
The categories of this paper are: q-bio
17
The categories of this paper are: q-fin
29
The categories of this paper are: funct-an