Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Distillation of Weighted Automata from Recurrent Neural Networks using a
  Spectral Approach   This paper is an attempt to bridge the gap between deep learning and
grammatical inference. Indeed, it provides an algorithm to extract a
(stochastic) formal language from any recurrent neural network trained for
language modelling. In detail, the algorithm uses the already trained network
as an oracle -- and thus does not require the access to the inner
representation of the black-box -- and applies a spectral approach to infer a
weighted automaton.
  As weighted automata compute linear functions, they are computationally more
efficient than neural networks and thus the nature of the approach is the one
of knowledge distillation. We detail experiments on 62 data sets (both
synthetic and from real-world applications) that allow an in-depth study of the
abilities of the proposed algorithm. The results show the WA we extract are
good approximations of the RNN, validating the approach. Moreover, we show how
the process provides interesting insights toward the behavior of RNN learned on
data, enlarging the scope of this work to the one of explainability of deep
learning models.

6
The categories of this paper are: astro-ph
11
The categories of this paper are: hep-lat
35
The categories of this paper are: mtrl-th
28
The categories of this paper are: alg-geom
9
The categories of this paper are: nlin
13
The categories of this paper are: quant-ph
33
The categories of this paper are: plasm-ph
36
The categories of this paper are: cmp-lg
29
The categories of this paper are: funct-an
3
The categories of this paper are: physics
14
The categories of this paper are: math-ph
12
The categories of this paper are: nucl-th
20
The categories of this paper are: acc-phys
32
The categories of this paper are: atom-ph
22
The categories of this paper are: chao-dyn
8
The categories of this paper are: hep-ex