Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: DADAM: A Consensus-based Distributed Adaptive Gradient Method for Online
  Optimization   Adaptive gradient-based optimization methods such as \textsc{Adagrad},
\textsc{Rmsprop}, and \textsc{Adam} are widely used in solving large-scale
machine learning problems including deep learning. A number of schemes have
been proposed in the literature aiming at parallelizing them, based on
communications of peripheral nodes with a central node, but incur high
communications cost. To address this issue, we develop a novel consensus-based
distributed adaptive moment estimation method (\textsc{Dadam}) for online
optimization over a decentralized network that enables data parallelization, as
well as decentralized computation. The method is particularly useful, since it
can accommodate settings where access to local data is allowed. Further, as
established theoretically in this work, it can outperform centralized adaptive
algorithms, for certain classes of loss functions used in applications. We
analyze the convergence properties of the proposed algorithm and provide a
dynamic regret bound on the convergence rate of adaptive moment estimation
methods in both stochastic and deterministic settings. Empirical results
demonstrate that \textsc{Dadam} works also well in practice and compares
favorably to competing online optimization methods.

13
The categories of this paper are: quant-ph
37
The categories of this paper are: supr-con
14
The categories of this paper are: math-ph
11
The categories of this paper are: hep-lat
15
The categories of this paper are: nucl-ex
34
The categories of this paper are: chem-ph
17
The categories of this paper are: q-fin
10
The categories of this paper are: q-bio
30
The categories of this paper are: q-alg
28
The categories of this paper are: alg-geom
32
The categories of this paper are: atom-ph
3
The categories of this paper are: physics
0
The categories of this paper are: hep-ph
9
The categories of this paper are: nlin
7
The categories of this paper are: hep-th
8
The categories of this paper are: hep-ex