Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Model compression as constrained optimization, with application to
  neural nets. Part V: combining compressions   Model compression is generally performed by using quantization, low-rank
approximation or pruning, for which various algorithms have been researched in
recent years. One fundamental question is: what types of compression work
better for a given model? Or even better: can we improve by combining
compressions in a suitable way? We formulate this generally as a problem of
optimizing the loss but where the weights are constrained to equal an additive
combination of separately compressed parts; and we give an algorithm to learn
the corresponding parts' parameters. Experimentally with deep neural nets, we
observe that 1) we can find significantly better models in the
error-compression space, indicating that different compression types have
complementary benefits, and 2) the best type of combination depends exquisitely
on the type of neural net. For example, we can compress ResNets and AlexNet
using only 1 bit per weight without error degradation at the cost of adding a
few floating point weights. However, VGG nets can be better compressed by
combining low-rank with a few floating point weights.

5
The categories of this paper are: gr-qc
22
The categories of this paper are: chao-dyn
14
The categories of this paper are: math-ph
26
The categories of this paper are: bayes-an
20
The categories of this paper are: acc-phys
9
The categories of this paper are: nlin
34
The categories of this paper are: chem-ph
8
The categories of this paper are: hep-ex
6
The categories of this paper are: astro-ph
24
The categories of this paper are: dg-ga
15
The categories of this paper are: nucl-ex
31
The categories of this paper are: ao-sci
11
The categories of this paper are: hep-lat
12
The categories of this paper are: nucl-th
4
The categories of this paper are: cond-mat
3
The categories of this paper are: physics