Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: LAWDR: Language-Agnostic Weighted Document Representations from
  Pre-trained Models   Cross-lingual document representations enable language understanding in
multilingual contexts and allow transfer learning from high-resource to
low-resource languages at the document level. Recently large pre-trained
language models such as BERT, XLM and XLM-RoBERTa have achieved great success
when fine-tuned on sentence-level downstream tasks. It is tempting to apply
these cross-lingual models to document representation learning. However, there
are two challenges: (1) these models impose high costs on long document
processing and thus many of them have strict length limit; (2) model
fine-tuning requires extra data and computational resources, which is not
practical in resource-limited settings. In this work, we address these
challenges by proposing unsupervised Language-Agnostic Weighted Document
Representations (LAWDR). We study the geometry of pre-trained sentence
embeddings and leverage it to derive document representations without
fine-tuning. Evaluated on cross-lingual document alignment, LAWDR demonstrates
comparable performance to state-of-the-art models on benchmark datasets.

34
The categories of this paper are: chem-ph
23
The categories of this paper are: solv-int
32
The categories of this paper are: atom-ph
13
The categories of this paper are: quant-ph
22
The categories of this paper are: chao-dyn
2
The categories of this paper are: math
31
The categories of this paper are: ao-sci
33
The categories of this paper are: plasm-ph
17
The categories of this paper are: q-fin
20
The categories of this paper are: acc-phys
6
The categories of this paper are: astro-ph
29
The categories of this paper are: funct-an
14
The categories of this paper are: math-ph
35
The categories of this paper are: mtrl-th
18
The categories of this paper are: econ
10
The categories of this paper are: q-bio