Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Sparsity Emerges Naturally in Neural Language Models   Concerns about interpretability, computational resources, and principled
inductive priors have motivated efforts to engineer sparse neural models for
NLP tasks. If sparsity is important for NLP, might well-trained neural models
naturally become roughly sparse? Using the Taxi-Euclidean norm to measure
sparsity, we find that frequent input words are associated with concentrated or
sparse activations, while frequent target words are associated with dispersed
activations but concentrated gradients. We find that gradients associated with
function words are more concentrated than the gradients of content words, even
controlling for word frequency.

31
The categories of this paper are: ao-sci
0
The categories of this paper are: hep-ph
32
The categories of this paper are: atom-ph
4
The categories of this paper are: cond-mat
14
The categories of this paper are: math-ph
26
The categories of this paper are: bayes-an
3
The categories of this paper are: physics
29
The categories of this paper are: funct-an
34
The categories of this paper are: chem-ph
8
The categories of this paper are: hep-ex
35
The categories of this paper are: mtrl-th
9
The categories of this paper are: nlin
36
The categories of this paper are: cmp-lg
21
The categories of this paper are: adap-org
15
The categories of this paper are: nucl-ex
25
The categories of this paper are: patt-sol