Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Structured Convolutions for Efficient Neural Network Design   In this work, we tackle model efficiency by exploiting redundancy in the
\textit{implicit structure} of the building blocks of convolutional neural
networks. We start our analysis by introducing a general definition of
Composite Kernel structures that enable the execution of convolution operations
in the form of efficient, scaled, sum-pooling components. As its special case,
we propose \textit{Structured Convolutions} and show that these allow
decomposition of the convolution operation into a sum-pooling operation
followed by a convolution with significantly lower complexity and fewer
weights. We show how this decomposition can be applied to 2D and 3D kernels as
well as the fully-connected layers. Furthermore, we present a Structural
Regularization loss that promotes neural network layers to leverage on this
desired structure in a way that, after training, they can be decomposed with
negligible performance loss. By applying our method to a wide range of CNN
architectures, we demonstrate "structured" versions of the ResNets that are up
to 2$\times$ smaller and a new Structured-MobileNetV2 that is more efficient
while staying within an accuracy loss of 1% on ImageNet and CIFAR-10 datasets.
We also show similar structured versions of EfficientNet on ImageNet and HRNet
architecture for semantic segmentation on the Cityscapes dataset. Our method
performs equally well or superior in terms of the complexity reduction in
comparison to the existing tensor decomposition and channel pruning methods.

8
The categories of this paper are: hep-ex
36
The categories of this paper are: cmp-lg
27
The categories of this paper are: comp-gas
30
The categories of this paper are: q-alg
17
The categories of this paper are: q-fin
14
The categories of this paper are: math-ph
9
The categories of this paper are: nlin
12
The categories of this paper are: nucl-th
37
The categories of this paper are: supr-con
23
The categories of this paper are: solv-int
19
The categories of this paper are: eess
18
The categories of this paper are: econ
35
The categories of this paper are: mtrl-th
4
The categories of this paper are: cond-mat
5
The categories of this paper are: gr-qc
2
The categories of this paper are: math