Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: A Weight Initialization Based on the Linear Product Structure for Neural
  Networks   Weight initialization plays an important role in training neural networks and
also affects tremendous deep learning applications. Various weight
initialization strategies have already been developed for different activation
functions with different neural networks. These initialization algorithms are
based on minimizing the variance of the parameters between layers and might
still fail when neural networks are deep, e.g., dying ReLU. To address this
challenge, we study neural networks from a nonlinear computation point of view
and propose a novel weight initialization strategy that is based on the linear
product structure (LPS) of neural networks. The proposed strategy is derived
from the polynomial approximation of activation functions by using theories of
numerical algebraic geometry to guarantee to find all the local minima. We also
provide a theoretical analysis that the LPS initialization has a lower
probability of dying ReLU comparing to other existing initialization
strategies. Finally, we test the LPS initialization algorithm on both fully
connected neural networks and convolutional neural networks to show its
feasibility, efficiency, and robustness on public datasets.

3
The categories of this paper are: physics
8
The categories of this paper are: hep-ex
37
The categories of this paper are: supr-con
21
The categories of this paper are: adap-org
26
The categories of this paper are: bayes-an
35
The categories of this paper are: mtrl-th
9
The categories of this paper are: nlin
10
The categories of this paper are: q-bio
16
The categories of this paper are: stat
4
The categories of this paper are: cond-mat
6
The categories of this paper are: astro-ph
30
The categories of this paper are: q-alg
24
The categories of this paper are: dg-ga
33
The categories of this paper are: plasm-ph
27
The categories of this paper are: comp-gas
23
The categories of this paper are: solv-int