Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: How Large a Vocabulary Does Text Classification Need? A Variational
  Approach to Vocabulary Selection   With the rapid development in deep learning, deep neural networks have been
widely adopted in many real-life natural language applications. Under deep
neural networks, a pre-defined vocabulary is required to vectorize text inputs.
The canonical approach to select pre-defined vocabulary is based on the word
frequency, where a threshold is selected to cut off the long tail distribution.
However, we observed that such simple approach could easily lead to under-sized
vocabulary or over-sized vocabulary issues. Therefore, we are interested in
understanding how the end-task classification accuracy is related to the
vocabulary size and what is the minimum required vocabulary size to achieve a
specific performance. In this paper, we provide a more sophisticated
variational vocabulary dropout (VVD) based on variational dropout to perform
vocabulary selection, which can intelligently select the subset of the
vocabulary to achieve the required performance. To evaluate different
algorithms on the newly proposed vocabulary selection problem, we propose two
new metrics: Area Under Accuracy-Vocab Curve and Vocab Size under X\% Accuracy
Drop. Through extensive experiments on various NLP classification tasks, our
variational framework is shown to significantly outperform the frequency-based
and other selection baselines on these metrics.

30
The categories of this paper are: q-alg
7
The categories of this paper are: hep-th
32
The categories of this paper are: atom-ph
9
The categories of this paper are: nlin
25
The categories of this paper are: patt-sol
11
The categories of this paper are: hep-lat
2
The categories of this paper are: math
35
The categories of this paper are: mtrl-th
14
The categories of this paper are: math-ph
22
The categories of this paper are: chao-dyn
5
The categories of this paper are: gr-qc
10
The categories of this paper are: q-bio
24
The categories of this paper are: dg-ga
29
The categories of this paper are: funct-an
4
The categories of this paper are: cond-mat
6
The categories of this paper are: astro-ph