Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Towards Efficient NLP: A Standard Evaluation and A Strong Baseline   Supersized pre-trained language models have pushed the accuracy of various
natural language processing (NLP) tasks to a new state-of-the-art (SOTA).
Rather than pursuing the reachless SOTA accuracy, more and more researchers
start paying attention on model efficiency and usability. Different from
accuracy, the metric for efficiency varies across different studies, making
them hard to be fairly compared. To that end, this work presents ELUE
(Efficient Language Understanding Evaluation), a standard evaluation, and a
public leaderboard for efficient NLP models. ELUE is dedicated to depict the
Pareto Frontier for various language understanding tasks, such that it can tell
whether and how much a method achieves Pareto improvement. Along with the
benchmark, we also release a strong baseline, ElasticBERT, which allows BERT to
exit at any layer in both static and dynamic ways. We demonstrate the
ElasticBERT, despite its simplicity, outperforms or performs on par with SOTA
compressed and early exiting models. With ElasticBERT, the proposed ELUE has a
strong Pareto Frontier and makes a better evaluation for efficient NLP models.

7
The categories of this paper are: hep-th
34
The categories of this paper are: chem-ph
33
The categories of this paper are: plasm-ph
28
The categories of this paper are: alg-geom
16
The categories of this paper are: stat
3
The categories of this paper are: physics
31
The categories of this paper are: ao-sci
19
The categories of this paper are: eess
37
The categories of this paper are: supr-con
8
The categories of this paper are: hep-ex
20
The categories of this paper are: acc-phys
23
The categories of this paper are: solv-int
4
The categories of this paper are: cond-mat
35
The categories of this paper are: mtrl-th
0
The categories of this paper are: hep-ph
12
The categories of this paper are: nucl-th