Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Estimating the Brittleness of AI: Safety Integrity Levels and the Need
  for Testing Out-Of-Distribution Performance   Test, Evaluation, Verification, and Validation (TEVV) for Artificial
Intelligence (AI) is a challenge that threatens to limit the economic and
societal rewards that AI researchers have devoted themselves to producing. A
central task of TEVV for AI is estimating brittleness, where brittleness
implies that the system functions well within some bounds and poorly outside of
those bounds. This paper argues that neither of those criteria are certain of
Deep Neural Networks. First, highly touted AI successes (eg. image
classification and speech recognition) are orders of magnitude more
failure-prone than are typically certified in critical systems even within
design bounds (perfectly in-distribution sampling). Second, performance falls
off only gradually as inputs become further Out-Of-Distribution (OOD). Enhanced
emphasis is needed on designing systems that are resilient despite
failure-prone AI components as well as on evaluating and improving OOD
performance in order to get AI to where it can clear the challenging hurdles of
TEVV and certification.

28
The categories of this paper are: alg-geom
4
The categories of this paper are: cond-mat
5
The categories of this paper are: gr-qc
24
The categories of this paper are: dg-ga
19
The categories of this paper are: eess
34
The categories of this paper are: chem-ph
21
The categories of this paper are: adap-org
26
The categories of this paper are: bayes-an
35
The categories of this paper are: mtrl-th
25
The categories of this paper are: patt-sol
15
The categories of this paper are: nucl-ex
10
The categories of this paper are: q-bio
27
The categories of this paper are: comp-gas
2
The categories of this paper are: math
33
The categories of this paper are: plasm-ph
6
The categories of this paper are: astro-ph