Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Empirical Frequentist Coverage of Deep Learning Uncertainty
  Quantification Procedures   Uncertainty quantification for complex deep learning models is increasingly
important as these techniques see growing use in high-stakes, real-world
settings. Currently, the quality of a model's uncertainty is evaluated using
point-prediction metrics such as negative log-likelihood or the Brier score on
heldout data. In this study, we provide the first large scale evaluation of the
empirical frequentist coverage properties of well known uncertainty
quantification techniques on a suite of regression and classification tasks. We
find that, in general, some methods do achieve desirable coverage properties on
in distribution samples, but that coverage is not maintained on
out-of-distribution data. Our results demonstrate the failings of current
uncertainty quantification techniques as dataset shift increases and establish
coverage as an important metric in developing models for real-world
applications.

25
The categories of this paper are: patt-sol
9
The categories of this paper are: nlin
28
The categories of this paper are: alg-geom
3
The categories of this paper are: physics
26
The categories of this paper are: bayes-an
5
The categories of this paper are: gr-qc
8
The categories of this paper are: hep-ex
35
The categories of this paper are: mtrl-th
14
The categories of this paper are: math-ph
24
The categories of this paper are: dg-ga
27
The categories of this paper are: comp-gas
12
The categories of this paper are: nucl-th
37
The categories of this paper are: supr-con
16
The categories of this paper are: stat
21
The categories of this paper are: adap-org
15
The categories of this paper are: nucl-ex