Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Learning Term Discrimination   Document indexing is a key component for efficient information retrieval
(IR). After preprocessing steps such as stemming and stop-word removal,
document indexes usually store term-frequencies (tf). Along with tf (that only
reflects the importance of a term in a document), traditional IR models use
term discrimination values (TDVs) such as inverse document frequency (idf) to
favor discriminative terms during retrieval. In this work, we propose to learn
TDVs for document indexing with shallow neural networks that approximate
traditional IR ranking functions such as TF-IDF and BM25. Our proposal
outperforms, both in terms of nDCG and recall, traditional approaches, even
with few positively labelled query-document pairs as learning data. Our learned
TDVs, when used to filter out terms of the vocabulary that have zero
discrimination value, allow to both significantly lower the memory footprint of
the inverted index and speed up the retrieval process (BM25 is up to 3~times
faster), without degrading retrieval quality.

34
The categories of this paper are: chem-ph
28
The categories of this paper are: alg-geom
8
The categories of this paper are: hep-ex
12
The categories of this paper are: nucl-th
3
The categories of this paper are: physics
32
The categories of this paper are: atom-ph
10
The categories of this paper are: q-bio
14
The categories of this paper are: math-ph
19
The categories of this paper are: eess
36
The categories of this paper are: cmp-lg
29
The categories of this paper are: funct-an
26
The categories of this paper are: bayes-an
2
The categories of this paper are: math
35
The categories of this paper are: mtrl-th
37
The categories of this paper are: supr-con
17
The categories of this paper are: q-fin