Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: When Semi-Supervised Learning Meets Transfer Learning: Training
  Strategies, Models and Datasets   Semi-Supervised Learning (SSL) has been proved to be an effective way to
leverage both labeled and unlabeled data at the same time. Recent
semi-supervised approaches focus on deep neural networks and have achieved
promising results on several benchmarks: CIFAR10, CIFAR100 and SVHN. However,
most of their experiments are based on models trained from scratch instead of
pre-trained models. On the other hand, transfer learning has demonstrated its
value when the target domain has limited labeled data. Here comes the intuitive
question: is it possible to incorporate SSL when fine-tuning a pre-trained
model? We comprehensively study how SSL methods starting from pretrained models
perform under varying conditions, including training strategies, architecture
choice and datasets. From this study, we obtain several interesting and useful
observations.
  While practitioners have had an intuitive understanding of these
observations, we do a comprehensive emperical analysis and demonstrate that:
(1) the gains from SSL techniques over a fully-supervised baseline are smaller
when trained from a pre-trained model than when trained from random
initialization, (2) when the domain of the source data used to train the
pre-trained model differs significantly from the domain of the target task, the
gains from SSL are significantly higher and (3) some SSL methods are able to
advance fully-supervised baselines (like Pseudo-Label).
  We hope our studies can deepen the understanding of SSL research and
facilitate the process of developing more effective SSL methods to utilize
pre-trained models. Code is now available at github.

23
The categories of this paper are: solv-int
33
The categories of this paper are: plasm-ph
18
The categories of this paper are: econ
10
The categories of this paper are: q-bio
4
The categories of this paper are: cond-mat
28
The categories of this paper are: alg-geom
6
The categories of this paper are: astro-ph
24
The categories of this paper are: dg-ga
37
The categories of this paper are: supr-con
17
The categories of this paper are: q-fin
13
The categories of this paper are: quant-ph
19
The categories of this paper are: eess
34
The categories of this paper are: chem-ph
31
The categories of this paper are: ao-sci
0
The categories of this paper are: hep-ph
26
The categories of this paper are: bayes-an