Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Randomized Stochastic Variance-Reduced Methods for Multi-Task Stochastic
  Bilevel Optimization   In this paper, we consider non-convex stochastic bilevel optimization (SBO)
problems that have many applications in machine learning. Although numerous
studies have proposed stochastic algorithms for solving these problems, they
are limited in two perspectives: (i) their sample complexities are high, which
do not match the state-of-the-art result for non-convex stochastic
optimization; (ii) their algorithms are tailored to problems with only one
lower-level problem. When there are many lower-level problems, it could be
prohibitive to process all these lower-level problems at each iteration. To
address these limitations, this paper proposes fast randomized stochastic
algorithms for non-convex SBO problems. First, we present a stochastic method
for non-convex SBO with only one lower problem and establish its sample
complexity of $O(1/\epsilon^3)$ for finding an $\epsilon$-stationary point
under Lipschitz continuous conditions of stochastic oracles, matching the lower
bound for stochastic smooth non-convex optimization. Second, we present a
randomized stochastic method for non-convex SBO with $m>1$ lower level problems
(multi-task SBO) by processing a constant number of lower problems at each
iteration, and establish its sample complexity no worse than $O(m/\epsilon^3)$,
which could be a better complexity than that of simply processing all $m$ lower
problems at each iteration. Lastly, we establish even faster convergence
results for gradient-dominant functions. To the best of our knowledge, this is
the first work considering multi-task SBO and developing state-of-the-art
sample complexity results.

20
The categories of this paper are: acc-phys
16
The categories of this paper are: stat
22
The categories of this paper are: chao-dyn
35
The categories of this paper are: mtrl-th
24
The categories of this paper are: dg-ga
36
The categories of this paper are: cmp-lg
15
The categories of this paper are: nucl-ex
34
The categories of this paper are: chem-ph
14
The categories of this paper are: math-ph
29
The categories of this paper are: funct-an
33
The categories of this paper are: plasm-ph
6
The categories of this paper are: astro-ph
4
The categories of this paper are: cond-mat
37
The categories of this paper are: supr-con
12
The categories of this paper are: nucl-th
19
The categories of this paper are: eess