Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Local Convolutions Cause an Implicit Bias towards High Frequency
  Adversarial Examples   Adversarial Attacks are still a significant challenge for neural networks.
Recent work has shown that adversarial perturbations typically contain
high-frequency features, but the root cause of this phenomenon remains unknown.
Inspired by theoretical work on linear full-width convolutional models, we
hypothesize that the local (i.e. bounded-width) convolutional operations
commonly used in current neural networks are implicitly biased to learn high
frequency features, and that this is one of the root causes of high frequency
adversarial examples. To test this hypothesis, we analyzed the impact of
different choices of linear and nonlinear architectures on the implicit bias of
the learned features and the adversarial perturbations, in both spatial and
frequency domains. We find that the high-frequency adversarial perturbations
are critically dependent on the convolution operation because the
spatially-limited nature of local convolutions induces an implicit bias towards
high frequency features. The explanation for the latter involves the Fourier
Uncertainty Principle: a spatially-limited (local in the space domain) filter
cannot also be frequency-limited (local in the frequency domain). Furthermore,
using larger convolution kernel sizes or avoiding convolutions (e.g. by using
Vision Transformers architecture) significantly reduces this high frequency
bias, but not the overall susceptibility to attacks. Looking forward, our work
strongly suggests that understanding and controlling the implicit bias of
architectures will be essential for achieving adversarial robustness.

35
The categories of this paper are: mtrl-th
27
The categories of this paper are: comp-gas
29
The categories of this paper are: funct-an
4
The categories of this paper are: cond-mat
11
The categories of this paper are: hep-lat
30
The categories of this paper are: q-alg
2
The categories of this paper are: math
14
The categories of this paper are: math-ph
10
The categories of this paper are: q-bio
15
The categories of this paper are: nucl-ex
26
The categories of this paper are: bayes-an
21
The categories of this paper are: adap-org
25
The categories of this paper are: patt-sol
24
The categories of this paper are: dg-ga
7
The categories of this paper are: hep-th
22
The categories of this paper are: chao-dyn