Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Optimization of Graph Neural Networks: Implicit Acceleration by Skip
  Connections and More Depth   Graph Neural Networks (GNNs) have been studied through the lens of expressive
power and generalization. However, their optimization properties are less well
understood. We take the first step towards analyzing GNN training by studying
the gradient dynamics of GNNs. First, we analyze linearized GNNs and prove that
despite the non-convexity of training, convergence to a global minimum at a
linear rate is guaranteed under mild assumptions that we validate on real-world
graphs. Second, we study what may affect the GNNs' training speed. Our results
show that the training of GNNs is implicitly accelerated by skip connections,
more depth, and/or a good label distribution. Empirical results confirm that
our theoretical results for linearized GNNs align with the training behavior of
nonlinear GNNs. Our results provide the first theoretical support for the
success of GNNs with skip connections in terms of optimization, and suggest
that deep GNNs with skip connections would be promising in practice.

9
The categories of this paper are: nlin
26
The categories of this paper are: bayes-an
10
The categories of this paper are: q-bio
36
The categories of this paper are: cmp-lg
31
The categories of this paper are: ao-sci
15
The categories of this paper are: nucl-ex
25
The categories of this paper are: patt-sol
17
The categories of this paper are: q-fin
20
The categories of this paper are: acc-phys
6
The categories of this paper are: astro-ph
5
The categories of this paper are: gr-qc
14
The categories of this paper are: math-ph
35
The categories of this paper are: mtrl-th
27
The categories of this paper are: comp-gas
7
The categories of this paper are: hep-th
32
The categories of this paper are: atom-ph