Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Smoothed Contrastive Learning for Unsupervised Sentence Embedding   Contrastive learning has been gradually applied to learn high-quality
unsupervised sentence embedding. Among the previous un-supervised methods, the
latest state-of-the-art method, as far as we know, is unsupervised SimCSE
(unsup-SimCSE). Unsup-SimCSE uses the InfoNCE1loss function in the training
stage by pulling semantically similar sentences together and pushing apart
dis-similar ones.Theoretically, we expect to use larger batches in unsup-SimCSE
to get more adequate comparisons among samples and avoid overfitting. However,
increasing the batch size does not always lead to improvements, but instead
even lead to performance degradation when the batch size exceeds a threshold.
Through statistical observation, we find that this is probably due to the
introduction of low-confidence negative pairs after in-creasing the batch size.
To alleviate this problem, we introduce a simple smoothing strategy upon the
InfoNCE loss function, termedGaussian Smoothing InfoNCE
(GS-InfoNCE).Specifically, we add random Gaussian noise vectors as negative
samples, which act asa smoothing of the negative sample space.Though being
simple, the proposed smooth-ing strategy brings substantial improvements to
unsup-SimCSE. We evaluate GS-InfoNCEon the standard semantic text similarity
(STS)task. GS-InfoNCE outperforms the state-of-the-art unsup-SimCSE by an
average Spear-man correlation of 1.38%, 0.72%, 1.17% and0.28% on the base of
BERT-base, BERT-large,RoBERTa-base and RoBERTa-large, respectively.

30
The categories of this paper are: q-alg
5
The categories of this paper are: gr-qc
35
The categories of this paper are: mtrl-th
16
The categories of this paper are: stat
0
The categories of this paper are: hep-ph
14
The categories of this paper are: math-ph
11
The categories of this paper are: hep-lat
4
The categories of this paper are: cond-mat
7
The categories of this paper are: hep-th
32
The categories of this paper are: atom-ph
15
The categories of this paper are: nucl-ex
18
The categories of this paper are: econ
27
The categories of this paper are: comp-gas
17
The categories of this paper are: q-fin
36
The categories of this paper are: cmp-lg
24
The categories of this paper are: dg-ga