Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Probing Linguistic Information For Logical Inference In Pre-trained
  Language Models   Progress in pre-trained language models has led to a surge of impressive
results on downstream tasks for natural language understanding. Recent work on
probing pre-trained language models uncovered a wide range of linguistic
properties encoded in their contextualized representations. However, it is
unclear whether they encode semantic knowledge that is crucial to symbolic
inference methods. We propose a methodology for probing linguistic information
for logical inference in pre-trained language model representations. Our
probing datasets cover a list of linguistic phenomena required by major
symbolic inference systems. We find that (i) pre-trained language models do
encode several types of linguistic information for inference, but there are
also some types of information that are weakly encoded, (ii) language models
can effectively learn missing linguistic information through fine-tuning.
Overall, our findings provide insights into which aspects of linguistic
information for logical inference do language models and their pre-training
procedures capture. Moreover, we have demonstrated language models' potential
as semantic and background knowledge bases for supporting symbolic inference
methods.

13
The categories of this paper are: quant-ph
4
The categories of this paper are: cond-mat
3
The categories of this paper are: physics
18
The categories of this paper are: econ
0
The categories of this paper are: hep-ph
17
The categories of this paper are: q-fin
19
The categories of this paper are: eess
5
The categories of this paper are: gr-qc
25
The categories of this paper are: patt-sol
31
The categories of this paper are: ao-sci
30
The categories of this paper are: q-alg
32
The categories of this paper are: atom-ph
23
The categories of this paper are: solv-int
24
The categories of this paper are: dg-ga
37
The categories of this paper are: supr-con
10
The categories of this paper are: q-bio