Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Deriving Differential Target Propagation from Iterating Approximate
  Inverses   We show that a particular form of target propagation, i.e., relying on
learned inverses of each layer, which is differential, i.e., where the target
is a small perturbation of the forward propagation, gives rise to an update
rule which corresponds to an approximate Gauss-Newton gradient-based
optimization, without requiring the manipulation or inversion of large
matrices. What is interesting is that this is more biologically plausible than
back-propagation yet may turn out to implicitly provide a stronger optimization
procedure. Extending difference target propagation, we consider several
iterative calculations based on local auto-encoders at each layer in order to
achieve more precise inversions for more accurate target propagation and we
show that these iterative procedures converge exponentially fast if the
auto-encoding function minus the identity function has a Lipschitz constant
smaller than one, i.e., the auto-encoder is coarsely succeeding at performing
an inversion. We also propose a way to normalize the changes at each layer to
take into account the relative influence of each layer on the output, so that
larger weight changes are done on more influential layers, like would happen in
ordinary back-propagation with gradient descent.

7
The categories of this paper are: hep-th
28
The categories of this paper are: alg-geom
31
The categories of this paper are: ao-sci
19
The categories of this paper are: eess
10
The categories of this paper are: q-bio
17
The categories of this paper are: q-fin
14
The categories of this paper are: math-ph
13
The categories of this paper are: quant-ph
21
The categories of this paper are: adap-org
22
The categories of this paper are: chao-dyn
27
The categories of this paper are: comp-gas
8
The categories of this paper are: hep-ex
33
The categories of this paper are: plasm-ph
6
The categories of this paper are: astro-ph
29
The categories of this paper are: funct-an
35
The categories of this paper are: mtrl-th