Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Keep the Caption Information: Preventing Shortcut Learning in
  Contrastive Image-Caption Retrieval   To train image-caption retrieval (ICR) methods, contrastive loss functions
are a common choice for optimization functions. Unfortunately, contrastive ICR
methods are vulnerable to learning shortcuts: decision rules that perform well
on the training data but fail to transfer to other testing conditions. We
introduce an approach to reduce shortcut feature representations for the ICR
task: latent target decoding (LTD). We add an additional decoder to the
learning framework to reconstruct the input caption, which prevents the image
and caption encoder from learning shortcut features. Instead of reconstructing
input captions in the input space, we decode the semantics of the caption in a
latent space. We implement the LTD objective as an optimization constraint, to
ensure that the reconstruction loss is below a threshold value while primarily
optimizing for the contrastive loss. Importantly, LTD does not depend on
additional training data or expensive (hard) negative mining strategies. Our
experiments show that, unlike reconstructing the input caption, LTD reduces
shortcut learning and improves generalizability by obtaining higher recall@k
and r-precision scores. Additionally, we show that the evaluation scores
benefit from implementing LTD as an optimization constraint instead of a dual
loss.

16
The categories of this paper are: stat
18
The categories of this paper are: econ
6
The categories of this paper are: astro-ph
7
The categories of this paper are: hep-th
33
The categories of this paper are: plasm-ph
15
The categories of this paper are: nucl-ex
0
The categories of this paper are: hep-ph
37
The categories of this paper are: supr-con
29
The categories of this paper are: funct-an
21
The categories of this paper are: adap-org
28
The categories of this paper are: alg-geom
34
The categories of this paper are: chem-ph
13
The categories of this paper are: quant-ph
8
The categories of this paper are: hep-ex
22
The categories of this paper are: chao-dyn
11
The categories of this paper are: hep-lat