Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: An Analysis of Unsupervised Pre-training in Light of Recent Advances   Convolutional neural networks perform well on object recognition because of a
number of recent advances: rectified linear units (ReLUs), data augmentation,
dropout, and large labelled datasets. Unsupervised data has been proposed as
another way to improve performance. Unfortunately, unsupervised pre-training is
not used by state-of-the-art methods leading to the following question: Is
unsupervised pre-training still useful given recent advances? If so, when? We
answer this in three parts: we 1) develop an unsupervised method that
incorporates ReLUs and recent unsupervised regularization techniques, 2)
analyze the benefits of unsupervised pre-training compared to data augmentation
and dropout on CIFAR-10 while varying the ratio of unsupervised to supervised
samples, 3) verify our findings on STL-10. We discover unsupervised
pre-training, as expected, helps when the ratio of unsupervised to supervised
samples is high, and surprisingly, hurts when the ratio is low. We also use
unsupervised pre-training with additional color augmentation to achieve near
state-of-the-art performance on STL-10.

30
The categories of this paper are: q-alg
35
The categories of this paper are: mtrl-th
29
The categories of this paper are: funct-an
25
The categories of this paper are: patt-sol
17
The categories of this paper are: q-fin
26
The categories of this paper are: bayes-an
21
The categories of this paper are: adap-org
19
The categories of this paper are: eess
31
The categories of this paper are: ao-sci
20
The categories of this paper are: acc-phys
6
The categories of this paper are: astro-ph
3
The categories of this paper are: physics
23
The categories of this paper are: solv-int
8
The categories of this paper are: hep-ex
2
The categories of this paper are: math
9
The categories of this paper are: nlin