Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Capacity allocation through neural network layers   Capacity analysis has been recently introduced as a way to analyze how linear
models distribute their modelling capacity across the input space. In this
paper, we extend the notion of capacity allocation to the case of neural
networks with non-linear layers. We show that under some hypotheses the problem
is equivalent to linear capacity allocation, within some extended input space
that factors in the non-linearities. We introduce the notion of layer
decoupling, which quantifies the degree to which a non-linear activation
decouples its outputs, and show that it plays a central role in capacity
allocation through layers. In the highly non-linear limit where decoupling is
total, we show that the propagation of capacity throughout the layers follows a
simple markovian rule, which turns into a diffusion PDE in the limit of deep
networks with residual layers. This allows us to recover some known results
about deep neural networks, such as the size of the effective receptive field,
or why ResNets avoid the shattering problem.

4
The categories of this paper are: cond-mat
31
The categories of this paper are: ao-sci
10
The categories of this paper are: q-bio
22
The categories of this paper are: chao-dyn
34
The categories of this paper are: chem-ph
9
The categories of this paper are: nlin
12
The categories of this paper are: nucl-th
26
The categories of this paper are: bayes-an
37
The categories of this paper are: supr-con
20
The categories of this paper are: acc-phys
32
The categories of this paper are: atom-ph
23
The categories of this paper are: solv-int
35
The categories of this paper are: mtrl-th
24
The categories of this paper are: dg-ga
15
The categories of this paper are: nucl-ex
19
The categories of this paper are: eess