Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Arch-Net: Model Distillation for Architecture Agnostic Model Deployment   Vast requirement of computation power of Deep Neural Networks is a major
hurdle to their real world applications. Many recent Application Specific
Integrated Circuit (ASIC) chips feature dedicated hardware support for Neural
Network Acceleration. However, as ASICs take multiple years to develop, they
are inevitably out-paced by the latest development in Neural Architecture
Research. For example, Transformer Networks do not have native support on many
popular chips, and hence are difficult to deploy. In this paper, we propose
Arch-Net, a family of Neural Networks made up of only operators efficiently
supported across most architectures of ASICs. When a Arch-Net is produced, less
common network constructs, like Layer Normalization and Embedding Layers, are
eliminated in a progressive manner through label-free Blockwise Model
Distillation, while performing sub-eight bit quantization at the same time to
maximize performance. Empirical results on machine translation and image
classification tasks confirm that we can transform latest developed Neural
Architectures into fast running and as-accurate Arch-Net, ready for deployment
on multiple mass-produced ASIC chips. The code will be available at
https://github.com/megvii-research/Arch-Net.

10
The categories of this paper are: q-bio
34
The categories of this paper are: chem-ph
23
The categories of this paper are: solv-int
27
The categories of this paper are: comp-gas
37
The categories of this paper are: supr-con
16
The categories of this paper are: stat
25
The categories of this paper are: patt-sol
22
The categories of this paper are: chao-dyn
17
The categories of this paper are: q-fin
13
The categories of this paper are: quant-ph
35
The categories of this paper are: mtrl-th
11
The categories of this paper are: hep-lat
4
The categories of this paper are: cond-mat
31
The categories of this paper are: ao-sci
29
The categories of this paper are: funct-an
14
The categories of this paper are: math-ph