Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Influence Paths for Characterizing Subject-Verb Number Agreement in LSTM
  Language Models   LSTM-based recurrent neural networks are the state-of-the-art for many
natural language processing (NLP) tasks. Despite their performance, it is
unclear whether, or how, LSTMs learn structural features of natural languages
such as subject-verb number agreement in English. Lacking this understanding,
the generality of LSTM performance on this task and their suitability for
related tasks remains uncertain. Further, errors cannot be properly attributed
to a lack of structural capability, training data omissions, or other
exceptional faults. We introduce *influence paths*, a causal account of
structural properties as carried by paths across gates and neurons of a
recurrent neural network. The approach refines the notion of influence (the
subject's grammatical number has influence on the grammatical number of the
subsequent verb) into a set of gate or neuron-level paths. The set localizes
and segments the concept (e.g., subject-verb agreement), its constituent
elements (e.g., the subject), and related or interfering elements (e.g.,
attractors). We exemplify the methodology on a widely-studied multi-layer LSTM
language model, demonstrating its accounting for subject-verb number agreement.
The results offer both a finer and a more complete view of an LSTM's handling
of this structural aspect of the English language than prior results based on
diagnostic classifiers and ablation.

23
The categories of this paper are: solv-int
11
The categories of this paper are: hep-lat
25
The categories of this paper are: patt-sol
16
The categories of this paper are: stat
10
The categories of this paper are: q-bio
24
The categories of this paper are: dg-ga
18
The categories of this paper are: econ
7
The categories of this paper are: hep-th
4
The categories of this paper are: cond-mat
5
The categories of this paper are: gr-qc
19
The categories of this paper are: eess
20
The categories of this paper are: acc-phys
27
The categories of this paper are: comp-gas
26
The categories of this paper are: bayes-an
32
The categories of this paper are: atom-ph
29
The categories of this paper are: funct-an