Batch Normalization (BatchNorm) has shown to be effective for improving and accelerating the training of deep neural networks. However, recently it has been shown that it is also vulnerable to adversarial perturbations. In this work, we aim to investigate the cause of adversarial vulnerability of the BatchNorm. We hypothesize that the use of different normalization statistics during training and inference (mini-batch statistics for training and moving average of these values at inference) is the main cause of this adversarial vulnerability in the BatchNorm layer. We empirically proved this by experiments on various neural network architectures and datasets. Furthermore, we introduce Robust Normalization (RobustNorm) and experimentally show that it is not only resilient to adversarial perturbation but also inherit the benefits of BatchNorm.
Investigation of how BatchNorm causes adversarial vulnerability and how to avoid it.  This paper addresses vulnerability to adversarial perturbations in BatchNorm, and proposes an alternative called RobustNorm, using min-max rescaling instead of normalization. This paper investigates the reason behind the vulnerability of BatchNorm and proposes Robust Normalization, a normalization method that achieves significantly better results under a variety of attack methods.