We formulate an information-based optimization problem for supervised classification. For invertible neural networks, the control of these information terms is passed down to the latent features and parameter matrix in the last fully connected layer, given that mutual information is invariant under invertible map.   We propose an objective function and prove that it solves the optimization problem. Our framework allows us to learn latent features in an more interpretable form while improving the classification performance. We perform extensive quantitative and qualitative experiments in comparison with the existing state-of-the-art classification models.
we propose a regularizer that improves the classification performance of neural networks the authors propose to train a model from a point of maximizing mutual information between the predictions and the true outputs, with a regularization term that minimizes irrelevant information while learning. Proposes to decompose the parameters into an invertible feature map F and a linear transformation w in the last layer to maximize mutual information I(Y, \hat{T}) while constraining irrelevant information