Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Robust Named Entity Recognition with Truecasing Pretraining   Although modern named entity recognition (NER) systems show impressive
performance on standard datasets, they perform poorly when presented with noisy
data. In particular, capitalization is a strong signal for entities in many
languages, and even state of the art models overfit to this feature, with
drastically lower performance on uncapitalized text. In this work, we address
the problem of robustness of NER systems in data with noisy or uncertain
casing, using a pretraining objective that predicts casing in text, or a
truecaser, leveraging unlabeled data. The pretrained truecaser is combined with
a standard BiLSTM-CRF model for NER by appending output distributions to
character embeddings. In experiments over several datasets of varying domain
and casing quality, we show that our new model improves performance in uncased
text, even adding value to uncased BERT embeddings. Our method achieves a new
state of the art on the WNUT17 shared task dataset.

14
The categories of this paper are: math-ph
5
The categories of this paper are: gr-qc
17
The categories of this paper are: q-fin
30
The categories of this paper are: q-alg
24
The categories of this paper are: dg-ga
35
The categories of this paper are: mtrl-th
19
The categories of this paper are: eess
33
The categories of this paper are: plasm-ph
9
The categories of this paper are: nlin
16
The categories of this paper are: stat
0
The categories of this paper are: hep-ph
22
The categories of this paper are: chao-dyn
13
The categories of this paper are: quant-ph
15
The categories of this paper are: nucl-ex
18
The categories of this paper are: econ
37
The categories of this paper are: supr-con