Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Exploiting Local and Global Features in Transformer-based Extreme
  Multi-label Text Classification   Extreme multi-label text classification (XMTC) is the task of tagging each
document with the relevant labels from a very large space of predefined
categories. Recently, large pre-trained Transformer models have made
significant performance improvements in XMTC, which typically use the embedding
of the special CLS token to represent the entire document semantics as a global
feature vector, and match it against candidate labels. However, we argue that
such a global feature vector may not be sufficient to represent different
granularity levels of semantics in the document, and that complementing it with
the local word-level features could bring additional gains. Based on this
insight, we propose an approach that combines both the local and global
features produced by Transformer models to improve the prediction power of the
classifier. Our experiments show that the proposed model either outperforms or
is comparable to the state-of-the-art methods on benchmark datasets.

4
The categories of this paper are: cond-mat
30
The categories of this paper are: q-alg
29
The categories of this paper are: funct-an
31
The categories of this paper are: ao-sci
5
The categories of this paper are: gr-qc
0
The categories of this paper are: hep-ph
12
The categories of this paper are: nucl-th
14
The categories of this paper are: math-ph
22
The categories of this paper are: chao-dyn
28
The categories of this paper are: alg-geom
36
The categories of this paper are: cmp-lg
27
The categories of this paper are: comp-gas
19
The categories of this paper are: eess
32
The categories of this paper are: atom-ph
18
The categories of this paper are: econ
23
The categories of this paper are: solv-int