Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Surrogate Gradients Design   Surrogate gradient (SG) training provides the possibility to quickly transfer
all the gains made in deep learning to neuromorphic computing and neuromorphic
processors, with the consequent reduction in energy consumption. Evidence
supports that training can be robust to the choice of SG shape, after an
extensive search of hyper-parameters. However, random or grid search of
hyper-parameters becomes exponentially unfeasible as we consider more
hyper-parameters. Moreover, every point in the search can itself be highly time
and energy consuming for large networks and large datasets. In this article we
show how complex tasks and networks are more sensitive to SG choice. Secondly,
we show how low dampening, high sharpness and low tail fatness are preferred.
Thirdly, we observe that Glorot Uniform initialization is generally preferred
by most SG choices, with variability in the results. We finally provide a
theoretical solution to reduce the need of extensive gridsearch, to find SG
shape and initializations that result in improved accuracy.

31
The categories of this paper are: ao-sci
26
The categories of this paper are: bayes-an
6
The categories of this paper are: astro-ph
17
The categories of this paper are: q-fin
9
The categories of this paper are: nlin
13
The categories of this paper are: quant-ph
12
The categories of this paper are: nucl-th
16
The categories of this paper are: stat
33
The categories of this paper are: plasm-ph
10
The categories of this paper are: q-bio
2
The categories of this paper are: math
4
The categories of this paper are: cond-mat
8
The categories of this paper are: hep-ex
5
The categories of this paper are: gr-qc
37
The categories of this paper are: supr-con
30
The categories of this paper are: q-alg