Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: From FreEM to D'AlemBERT: a Large Corpus and a Language Model for Early
  Modern French   Language models for historical states of language are becoming increasingly
important to allow the optimal digitisation and analysis of old textual
sources. Because these historical states are at the same time more complex to
process and more scarce in the corpora available, specific efforts are
necessary to train natural language processing (NLP) tools adapted to the data.
In this paper, we present our efforts to develop NLP tools for Early Modern
French (historical French from the 16$^\text{th}$ to the 18$^\text{th}$
centuries). We present the $\text{FreEM}_{\text{max}}$ corpus of Early Modern
French and D'AlemBERT, a RoBERTa-based language model trained on
$\text{FreEM}_{\text{max}}$. We evaluate the usefulness of D'AlemBERT by
fine-tuning it on a part-of-speech tagging task, outperforming previous work on
the test set. Importantly, we find evidence for the transfer learning capacity
of the language model, since its performance on lesser-resourced time periods
appears to have been boosted by the more resourced ones. We release D'AlemBERT
and the open-sourced subpart of the $\text{FreEM}_{\text{max}}$ corpus.

28
The categories of this paper are: alg-geom
12
The categories of this paper are: nucl-th
15
The categories of this paper are: nucl-ex
0
The categories of this paper are: hep-ph
20
The categories of this paper are: acc-phys
6
The categories of this paper are: astro-ph
29
The categories of this paper are: funct-an
24
The categories of this paper are: dg-ga
14
The categories of this paper are: math-ph
10
The categories of this paper are: q-bio
26
The categories of this paper are: bayes-an
27
The categories of this paper are: comp-gas
36
The categories of this paper are: cmp-lg
4
The categories of this paper are: cond-mat
23
The categories of this paper are: solv-int
7
The categories of this paper are: hep-th