In this paper, the preparation of a neural network for pruning and few-bit quantization is formulated as a variational inference problem. To this end, a quantizing prior that leads to a multi-modal, sparse posterior distribution over weights, is introduced and a differentiable Kullback-Leibler divergence approximation for this prior is derived. After training with Variational Network Quantization, weights can be replaced by deterministic quantization values with small to negligible loss of task accuracy (including pruning by setting weights to 0). The method does not require fine-tuning after quantization. Results are shown for ternary quantization on LeNet-5 (MNIST) and DenseNet (CIFAR-10).
We quantize and prune neural network weights using variational Bayesian inference with a multi-modal, sparsity inducing prior. Proposes to use a mixture of continuous spike propto 1/abs as prior for a Bayesian neural network and demonstrates the good performance with relatively sparsified convnets for minist and cifar-10. This paper presents a variational Bayesian approach for quantising neural network weights to ternary values post-training in a principled way.