Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Learning Universal Representations from Word to Sentence   Despite the well-developed cut-edge representation learning for language,
most language representation models usually focus on specific level of
linguistic unit, which cause great inconvenience when being confronted with
handling multiple layers of linguistic objects in a unified way. Thus this work
introduces and explores the universal representation learning, i.e., embeddings
of different levels of linguistic unit in a uniform vector space through a
task-independent evaluation. We present our approach of constructing analogy
datasets in terms of words, phrases and sentences and experiment with multiple
representation models to examine geometric properties of the learned vector
space. Then we empirically verify that well pre-trained Transformer models
incorporated with appropriate training settings may effectively yield universal
representation. Especially, our implementation of fine-tuning ALBERT on NLI and
PPDB datasets achieves the highest accuracy on analogy tasks in different
language levels. Further experiments on the insurance FAQ task show
effectiveness of universal representation models in real-world applications.

13
The categories of this paper are: quant-ph
26
The categories of this paper are: bayes-an
3
The categories of this paper are: physics
0
The categories of this paper are: hep-ph
33
The categories of this paper are: plasm-ph
9
The categories of this paper are: nlin
32
The categories of this paper are: atom-ph
11
The categories of this paper are: hep-lat
4
The categories of this paper are: cond-mat
27
The categories of this paper are: comp-gas
17
The categories of this paper are: q-fin
14
The categories of this paper are: math-ph
21
The categories of this paper are: adap-org
10
The categories of this paper are: q-bio
16
The categories of this paper are: stat
20
The categories of this paper are: acc-phys