Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: ZO-AdaMM: Zeroth-Order Adaptive Momentum Method for Black-Box
  Optimization   The adaptive momentum method (AdaMM), which uses past gradients to update
descent directions and learning rates simultaneously, has become one of the
most popular first-order optimization methods for solving machine learning
problems. However, AdaMM is not suited for solving black-box optimization
problems, where explicit gradient forms are difficult or infeasible to obtain.
In this paper, we propose a zeroth-order AdaMM (ZO-AdaMM) algorithm, that
generalizes AdaMM to the gradient-free regime. We show that the convergence
rate of ZO-AdaMM for both convex and nonconvex optimization is roughly a factor
of $O(\sqrt{d})$ worse than that of the first-order AdaMM algorithm, where $d$
is problem size. In particular, we provide a deep understanding on why
Mahalanobis distance matters in convergence of ZO-AdaMM and other AdaMM-type
methods. As a byproduct, our analysis makes the first step toward understanding
adaptive learning rate methods for nonconvex constrained optimization.
Furthermore, we demonstrate two applications, designing per-image and universal
adversarial attacks from black-box neural networks, respectively. We perform
extensive experiments on ImageNet and empirically show that ZO-AdaMM converges
much faster to a solution of high accuracy compared with $6$ state-of-the-art
ZO optimization methods.

18
The categories of this paper are: econ
34
The categories of this paper are: chem-ph
0
The categories of this paper are: hep-ph
37
The categories of this paper are: supr-con
4
The categories of this paper are: cond-mat
10
The categories of this paper are: q-bio
14
The categories of this paper are: math-ph
7
The categories of this paper are: hep-th
17
The categories of this paper are: q-fin
36
The categories of this paper are: cmp-lg
23
The categories of this paper are: solv-int
22
The categories of this paper are: chao-dyn
9
The categories of this paper are: nlin
29
The categories of this paper are: funct-an
27
The categories of this paper are: comp-gas
32
The categories of this paper are: atom-ph