Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: A Sharp Estimate on the Transient Time of Distributed Stochastic
  Gradient Descent   This paper is concerned with minimizing the average of $n$ cost functions
over a network in which agents may communicate and exchange information with
each other. We consider the setting where only noisy gradient information is
available. To solve the problem, we study the distributed stochastic gradient
descent (DSGD) method and perform a non-asymptotic convergence analysis. For
strongly convex and smooth objective functions, DSGD asymptotically achieves
the optimal network independent convergence rate compared to centralized
stochastic gradient descent (SGD). Our main contribution is to characterize the
transient time needed for DSGD to approach the asymptotic convergence rate,
which we show behaves as $K_T=\mathcal{O}\left(\frac{n}{(1-\rho_w)^2}\right)$,
where $1-\rho_w$ denotes the spectral gap of the mixing matrix. Moreover, we
construct a "hard" optimization problem for which we show the transient time
needed for DSGD to approach the asymptotic convergence rate is lower bounded by
$\Omega \left(\frac{n}{(1-\rho_w)^2} \right)$, implying the sharpness of the
obtained result. Numerical experiments demonstrate the tightness of the
theoretical results.

8
The categories of this paper are: hep-ex
19
The categories of this paper are: eess
14
The categories of this paper are: math-ph
9
The categories of this paper are: nlin
36
The categories of this paper are: cmp-lg
6
The categories of this paper are: astro-ph
22
The categories of this paper are: chao-dyn
0
The categories of this paper are: hep-ph
23
The categories of this paper are: solv-int
37
The categories of this paper are: supr-con
24
The categories of this paper are: dg-ga
26
The categories of this paper are: bayes-an
35
The categories of this paper are: mtrl-th
4
The categories of this paper are: cond-mat
18
The categories of this paper are: econ
12
The categories of this paper are: nucl-th