Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Reparametrizing gradient descent   In this work, we propose an optimization algorithm which we call norm-adapted
gradient descent. This algorithm is similar to other gradient-based
optimization algorithms like Adam or Adagrad in that it adapts the learning
rate of stochastic gradient descent at each iteration. However, rather than
using statistical properties of observed gradients, norm-adapted gradient
descent relies on a first-order estimate of the effect of a standard gradient
descent update step, much like the Newton-Raphson method in many dimensions.
Our algorithm can also be compared to quasi-Newton methods, but we seek roots
rather than stationary points. Seeking roots can be justified by the fact that
for models with sufficient capacity measured by nonnegative loss functions,
roots coincide with global optima. This work presents several experiments where
we have used our algorithm; in these results, it appears norm-adapted descent
is particularly strong in regression settings but is also capable of training
classifiers.

15
The categories of this paper are: nucl-ex
0
The categories of this paper are: hep-ph
29
The categories of this paper are: funct-an
9
The categories of this paper are: nlin
14
The categories of this paper are: math-ph
21
The categories of this paper are: adap-org
28
The categories of this paper are: alg-geom
18
The categories of this paper are: econ
26
The categories of this paper are: bayes-an
6
The categories of this paper are: astro-ph
12
The categories of this paper are: nucl-th
22
The categories of this paper are: chao-dyn
30
The categories of this paper are: q-alg
24
The categories of this paper are: dg-ga
35
The categories of this paper are: mtrl-th
8
The categories of this paper are: hep-ex