Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Heterogeneous Calibration: A post-hoc model-agnostic framework for
  improved generalization   We introduce the notion of heterogeneous calibration that applies a post-hoc
model-agnostic transformation to model outputs for improving AUC performance on
binary classification tasks. We consider overconfident models, whose
performance is significantly better on training vs test data and give intuition
onto why they might under-utilize moderately effective simple patterns in the
data. We refer to these simple patterns as heterogeneous partitions of the
feature space and show theoretically that perfectly calibrating each partition
separately optimizes AUC. This gives a general paradigm of heterogeneous
calibration as a post-hoc procedure by which heterogeneous partitions of the
feature space are identified through tree-based algorithms and post-hoc
calibration techniques are applied to each partition to improve AUC. While the
theoretical optimality of this framework holds for any model, we focus on deep
neural networks (DNNs) and test the simplest instantiation of this paradigm on
a variety of open-source datasets. Experiments demonstrate the effectiveness of
this framework and the future potential for applying higher-performing
partitioning schemes along with more effective calibration techniques.

24
The categories of this paper are: dg-ga
13
The categories of this paper are: quant-ph
8
The categories of this paper are: hep-ex
17
The categories of this paper are: q-fin
34
The categories of this paper are: chem-ph
23
The categories of this paper are: solv-int
3
The categories of this paper are: physics
7
The categories of this paper are: hep-th
33
The categories of this paper are: plasm-ph
25
The categories of this paper are: patt-sol
32
The categories of this paper are: atom-ph
37
The categories of this paper are: supr-con
14
The categories of this paper are: math-ph
6
The categories of this paper are: astro-ph
12
The categories of this paper are: nucl-th
35
The categories of this paper are: mtrl-th