Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Diversify Your Datasets: Analyzing Generalization via Controlled
  Variance in Adversarial Datasets   Phenomenon-specific "adversarial" datasets have been recently designed to
perform targeted stress-tests for particular inference types. Recent work (Liu
et al., 2019a) proposed that such datasets can be utilized for training NLI and
other types of models, often allowing to learn the phenomenon in focus and
improve on the challenge dataset, indicating a "blind spot" in the original
training data. Yet, although a model can improve in such a training process, it
might still be vulnerable to other challenge datasets targeting the same
phenomenon but drawn from a different distribution, such as having a different
syntactic complexity level. In this work, we extend this method to drive
conclusions about a model's ability to learn and generalize a target phenomenon
rather than to "learn" a dataset, by controlling additional aspects in the
adversarial datasets. We demonstrate our approach on two inference phenomena -
dative alternation and numerical reasoning, elaborating, and in some cases
contradicting, the results of Liu et al.. Our methodology enables building
better challenge datasets for creating more robust models, and may yield better
model understanding and subsequent overarching improvements.

27
The categories of this paper are: comp-gas
2
The categories of this paper are: math
17
The categories of this paper are: q-fin
34
The categories of this paper are: chem-ph
0
The categories of this paper are: hep-ph
6
The categories of this paper are: astro-ph
10
The categories of this paper are: q-bio
22
The categories of this paper are: chao-dyn
7
The categories of this paper are: hep-th
9
The categories of this paper are: nlin
16
The categories of this paper are: stat
11
The categories of this paper are: hep-lat
19
The categories of this paper are: eess
29
The categories of this paper are: funct-an
32
The categories of this paper are: atom-ph
24
The categories of this paper are: dg-ga