Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Improving Transformation Invariance in Contrastive Representation
  Learning   We propose methods to strengthen the invariance properties of representations
obtained by contrastive learning. While existing approaches implicitly induce a
degree of invariance as representations are learned, we look to more directly
enforce invariance in the encoding process. To this end, we first introduce a
training objective for contrastive learning that uses a novel regularizer to
control how the representation changes under transformation. We show that
representations trained with this objective perform better on downstream tasks
and are more robust to the introduction of nuisance transformations at test
time. Second, we propose a change to how test time representations are
generated by introducing a feature averaging approach that combines encodings
from multiple transformations of the original input, finding that this leads to
across the board performance gains. Finally, we introduce the novel Spirograph
dataset to explore our ideas in the context of a differentiable generative
process with multiple downstream tasks, showing that our techniques for
learning invariance are highly beneficial.

36
The categories of this paper are: cmp-lg
24
The categories of this paper are: dg-ga
22
The categories of this paper are: chao-dyn
10
The categories of this paper are: q-bio
33
The categories of this paper are: plasm-ph
17
The categories of this paper are: q-fin
5
The categories of this paper are: gr-qc
29
The categories of this paper are: funct-an
3
The categories of this paper are: physics
30
The categories of this paper are: q-alg
20
The categories of this paper are: acc-phys
14
The categories of this paper are: math-ph
27
The categories of this paper are: comp-gas
28
The categories of this paper are: alg-geom
12
The categories of this paper are: nucl-th
0
The categories of this paper are: hep-ph