Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Improved Analysis and Rates for Variance Reduction under
  Without-replacement Sampling Orders   When applying a stochastic algorithm, one must choose an order to draw
samples. The practical choices are without-replacement sampling orders, which
are empirically faster and more cache-friendly than uniform-iid-sampling but
often have inferior theoretical guarantees. Without-replacement sampling is
well understood only for SGD without variance reduction. In this paper, we will
improve the convergence analysis and rates of variance reduction under
without-replacement sampling orders for composite finite-sum minimization.
  Our results are in two-folds. First, we develop a damped variant of Finito
called Prox-DFinito and establish its convergence rates with random
reshuffling, cyclic sampling, and shuffling-once, under both convex and
strongly convex scenarios. These rates match full-batch gradient descent and
are state-of-the-art compared to the existing results for without-replacement
sampling with variance-reduction. Second, our analysis can gauge how the cyclic
order will influence the rate of cyclic sampling and, thus, allows us to derive
the optimal fixed ordering. In the highly data-heterogeneous scenario,
Prox-DFinito with optimal cyclic sampling can attain a sample-size-independent
convergence rate, which, to our knowledge, is the first result that can match
with uniform-iid-sampling with variance reduction. We also propose a practical
method to discover the optimal cyclic ordering numerically.

17
The categories of this paper are: q-fin
14
The categories of this paper are: math-ph
32
The categories of this paper are: atom-ph
9
The categories of this paper are: nlin
0
The categories of this paper are: hep-ph
16
The categories of this paper are: stat
8
The categories of this paper are: hep-ex
29
The categories of this paper are: funct-an
36
The categories of this paper are: cmp-lg
31
The categories of this paper are: ao-sci
19
The categories of this paper are: eess
33
The categories of this paper are: plasm-ph
20
The categories of this paper are: acc-phys
25
The categories of this paper are: patt-sol
28
The categories of this paper are: alg-geom
12
The categories of this paper are: nucl-th