Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: On Information Plane Analyses of Neural Network Classifiers -- A Review   We review the current literature concerned with information plane analyses of
neural network classifiers. While the underlying information bottleneck theory
and the claim that information-theoretic compression is causally linked to
generalization are plausible, empirical evidence was found to be both
supporting and conflicting. We review this evidence together with a detailed
analysis of how the respective information quantities were estimated. Our
survey suggests that compression visualized in information planes is not
necessarily information-theoretic, but is rather often compatible with
geometric compression of the latent representations. This insight gives the
information plane a renewed justification.
  Aside from this, we shed light on the problem of estimating mutual
information in deterministic neural networks and its consequences.
Specifically, we argue that even in feed-forward neural networks the data
processing inequality need not hold for estimates of mutual information.
Similarly, while a fitting phase, in which the mutual information between the
latent representation and the target increases, is necessary (but not
sufficient) for good classification performance, depending on the specifics of
mutual information estimation such a fitting phase need not be visible in the
information plane.

34
The categories of this paper are: chem-ph
28
The categories of this paper are: alg-geom
20
The categories of this paper are: acc-phys
15
The categories of this paper are: nucl-ex
27
The categories of this paper are: comp-gas
21
The categories of this paper are: adap-org
37
The categories of this paper are: supr-con
13
The categories of this paper are: quant-ph
14
The categories of this paper are: math-ph
32
The categories of this paper are: atom-ph
29
The categories of this paper are: funct-an
36
The categories of this paper are: cmp-lg
10
The categories of this paper are: q-bio
4
The categories of this paper are: cond-mat
33
The categories of this paper are: plasm-ph
19
The categories of this paper are: eess