Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Second-order optimization for tensors with fixed tensor-train rank   There are several different notions of "low rank" for tensors, associated to
different formats. Among them, the Tensor Train (TT) format is particularly
well suited for tensors of high order, as it circumvents the curse of
dimensionality: an appreciable property for certain high-dimensional
applications. It is often convenient to model such applications as optimization
over the set of tensors with fixed (and low) TT rank. That set is a smooth
manifold. Exploiting this fact, others have shown that Riemannian optimization
techniques can perform particularly well on tasks such as tensor completion and
special large-scale linear systems from PDEs. So far, however, these
optimization techniques have been limited to first-order methods, likely
because of the technical hurdles in deriving exact expressions for the
Riemannian Hessian. In this paper, we derive a formula and efficient algorithm
to compute the Riemannian Hessian on this manifold. This allows us to implement
second-order optimization algorithms (namely, the Riemannian trust-region
method) and to analyze the conditioning of optimization problems over the fixed
TT rank manifold. In settings of interest, we show improved optimization
performance on tensor completion compared to first-order methods and
alternating least squares (ALS). Our work could have applications in training
of neural networks with tensor layers. Our code is freely available.

36
The categories of this paper are: cmp-lg
18
The categories of this paper are: econ
21
The categories of this paper are: adap-org
27
The categories of this paper are: comp-gas
15
The categories of this paper are: nucl-ex
6
The categories of this paper are: astro-ph
9
The categories of this paper are: nlin
8
The categories of this paper are: hep-ex
26
The categories of this paper are: bayes-an
29
The categories of this paper are: funct-an
16
The categories of this paper are: stat
13
The categories of this paper are: quant-ph
37
The categories of this paper are: supr-con
31
The categories of this paper are: ao-sci
0
The categories of this paper are: hep-ph
10
The categories of this paper are: q-bio