Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: The Low-Dimensional Linear Geometry of Contextualized Word
  Representations   Black-box probing models can reliably extract linguistic features like tense,
number, and syntactic role from pretrained word representations. However, the
manner in which these features are encoded in representations remains poorly
understood. We present a systematic study of the linear geometry of
contextualized word representations in ELMO and BERT. We show that a variety of
linguistic features (including structured dependency relationships) are encoded
in low-dimensional subspaces. We then refine this geometric picture, showing
that there are hierarchical relations between the subspaces encoding general
linguistic categories and more specific ones, and that low-dimensional feature
encodings are distributed rather than aligned to individual neurons. Finally,
we demonstrate that these linear subspaces are causally related to model
behavior, and can be used to perform fine-grained manipulation of BERT's output
distribution.

26
The categories of this paper are: bayes-an
29
The categories of this paper are: funct-an
36
The categories of this paper are: cmp-lg
25
The categories of this paper are: patt-sol
21
The categories of this paper are: adap-org
8
The categories of this paper are: hep-ex
13
The categories of this paper are: quant-ph
6
The categories of this paper are: astro-ph
24
The categories of this paper are: dg-ga
11
The categories of this paper are: hep-lat
14
The categories of this paper are: math-ph
34
The categories of this paper are: chem-ph
3
The categories of this paper are: physics
30
The categories of this paper are: q-alg
7
The categories of this paper are: hep-th
18
The categories of this paper are: econ