Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Towards the Generalization of Contrastive Self-Supervised Learning   Recently, self-supervised learning has attracted great attention, since it
only requires unlabeled data for training. Contrastive learning is a popular
approach for self-supervised learning and achieves promising empirical
performance. However, the theoretical understanding of its generalization
ability is still limited. To this end, we define a kind of
$(\sigma,\delta)$-measure to mathematically quantify the data augmentation, and
then provide an upper bound of the downstream classification error based on the
measure. We show that the generalization ability of contrastive self-supervised
learning depends on three key factors: alignment of positive samples,
divergence of class centers, and concentration of augmented data. The first two
factors can be optimized by contrastive algorithms, while the third one is
priorly determined by pre-defined data augmentation. With the above theoretical
findings, we further study two canonical contrastive losses, InfoNCE and
cross-correlation loss, and prove that both of them are able to obtain the
embedding space satisfying the aforementioned factors. Finally, we conduct
various experiments on the real-world dataset, and show that our theoretical
inferences on the relationship between the data augmentation and the
generalization of contrastive self-supervised learning agree with the empirical
observations.

10
The categories of this paper are: q-bio
0
The categories of this paper are: hep-ph
15
The categories of this paper are: nucl-ex
37
The categories of this paper are: supr-con
9
The categories of this paper are: nlin
3
The categories of this paper are: physics
22
The categories of this paper are: chao-dyn
35
The categories of this paper are: mtrl-th
7
The categories of this paper are: hep-th
26
The categories of this paper are: bayes-an
14
The categories of this paper are: math-ph
27
The categories of this paper are: comp-gas
31
The categories of this paper are: ao-sci
24
The categories of this paper are: dg-ga
23
The categories of this paper are: solv-int
8
The categories of this paper are: hep-ex