Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Building Chinese Biomedical Language Models via Multi-Level Text
  Discrimination   Pre-trained language models (PLMs), such as BERT and GPT, have revolutionized
the field of NLP, not only in the general domain but also in the biomedical
domain. Most prior efforts in building biomedical PLMs have resorted simply to
domain adaptation and focused mainly on English. In this work we introduce
eHealth, a Chinese biomedical PLM built from scratch with a new pre-training
framework. This new framework pre-trains eHealth as a discriminator through
both token- and sequence-level discrimination. The former is to detect input
tokens corrupted by a generator and recover their original identities from
plausible candidates, while the latter is to further distinguish corruptions of
a same original sequence from those of others. As such, eHealth can learn
language semantics at both token and sequence levels. Extensive experiments on
11 Chinese biomedical language understanding tasks of various forms verify the
effectiveness and superiority of our approach. We release the pre-trained model
at \url{https://github.com/PaddlePaddle/Research/tree/master/KG/eHealth} and
will also release the code later.

18
The categories of this paper are: econ
10
The categories of this paper are: q-bio
33
The categories of this paper are: plasm-ph
32
The categories of this paper are: atom-ph
25
The categories of this paper are: patt-sol
4
The categories of this paper are: cond-mat
0
The categories of this paper are: hep-ph
3
The categories of this paper are: physics
34
The categories of this paper are: chem-ph
24
The categories of this paper are: dg-ga
12
The categories of this paper are: nucl-th
9
The categories of this paper are: nlin
35
The categories of this paper are: mtrl-th
36
The categories of this paper are: cmp-lg
22
The categories of this paper are: chao-dyn
26
The categories of this paper are: bayes-an