Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: BARThez: a Skilled Pretrained French Sequence-to-Sequence Model   Inductive transfer learning has taken the entire NLP field by storm, with
models such as BERT and BART setting new state of the art on countless NLU
tasks. However, most of the available models and research have been conducted
for English. In this work, we introduce BARThez, the first large-scale
pretrained seq2seq model for French. Being based on BART, BARThez is
particularly well-suited for generative tasks. We evaluate BARThez on five
discriminative tasks from the FLUE benchmark and two generative tasks from a
novel summarization dataset, OrangeSum, that we created for this research. We
show BARThez to be very competitive with state-of-the-art BERT-based French
language models such as CamemBERT and FlauBERT. We also continue the
pretraining of a multilingual BART on BARThez' corpus, and show our resulting
model, mBARThez, to significantly boost BARThez' generative performance. Code,
data and models are publicly available.

34
The categories of this paper are: chem-ph
8
The categories of this paper are: hep-ex
16
The categories of this paper are: stat
20
The categories of this paper are: acc-phys
7
The categories of this paper are: hep-th
12
The categories of this paper are: nucl-th
32
The categories of this paper are: atom-ph
27
The categories of this paper are: comp-gas
15
The categories of this paper are: nucl-ex
5
The categories of this paper are: gr-qc
26
The categories of this paper are: bayes-an
17
The categories of this paper are: q-fin
31
The categories of this paper are: ao-sci
24
The categories of this paper are: dg-ga
22
The categories of this paper are: chao-dyn
30
The categories of this paper are: q-alg