Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Asymptotic Analysis via Stochastic Differential Equations of Gradient
  Descent Algorithms in Statistical and Computational Paradigms   This paper investigates asymptotic behaviors of gradient descent algorithms
(particularly accelerated gradient descent and stochastic gradient descent) in
the context of stochastic optimization arising in statistics and machine
learning where objective functions are estimated from available data. We show
that these algorithms can be computationally modeled by continuous-time
ordinary or stochastic differential equations. We establish gradient flow
central limit theorems to describe the limiting dynamic behaviors of these
computational algorithms and the large-sample performances of the related
statistical procedures, as the number of algorithm iterations and data size
both go to infinity, where the gradient flow central limit theorems are
governed by some linear ordinary or stochastic differential equations like
time-dependent Ornstein-Uhlenbeck processes. We illustrate that our study can
provide a novel unified framework for a joint computational and statistical
asymptotic analysis, where the computational asymptotic analysis studies
dynamic behaviors of these algorithms with the time (or the number of
iterations in the algorithms), the statistical asymptotic analysis investigates
large sample behaviors of the statistical procedures (like estimators and
classifiers) that the algorithms are applied to compute, and in fact the
statistical procedures are equal to the limits of the random sequences
generated from these iterative algorithms as the number of iterations goes to
infinity. The joint analysis results based on the obtained gradient flow
central limit theorems can identify four factors - learning rate, batch size,
gradient covariance, and Hessian - to derive new theory regarding the local
minima found by stochastic gradient descent for solving non-convex optimization
problems.

1
The categories of this paper are: cs
0
The categories of this paper are: hep-ph
12
The categories of this paper are: nucl-th
29
The categories of this paper are: funct-an
13
The categories of this paper are: quant-ph
21
The categories of this paper are: adap-org
11
The categories of this paper are: hep-lat
37
The categories of this paper are: supr-con
31
The categories of this paper are: ao-sci
34
The categories of this paper are: chem-ph
33
The categories of this paper are: plasm-ph
25
The categories of this paper are: patt-sol
2
The categories of this paper are: math
22
The categories of this paper are: chao-dyn
28
The categories of this paper are: alg-geom
5
The categories of this paper are: gr-qc