Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Out-of-Distribution Generalization in Kernel Regression   In real word applications, data generating process for training a machine
learning model often differs from what the model encounters in the test stage.
Understanding how and whether machine learning models generalize under such
distributional shifts have been a theoretical challenge. Here, we study
generalization in kernel regression when the training and test distributions
are different using methods from statistical physics. Using the replica method,
we derive an analytical formula for the out-of-distribution generalization
error applicable to any kernel and real datasets. We identify an overlap matrix
that quantifies the mismatch between distributions for a given kernel as a key
determinant of generalization performance under distribution shift. Using our
analytical expressions we elucidate various generalization phenomena including
possible improvement in generalization when there is a mismatch. We develop
procedures for optimizing training and test distributions for a given data
budget to find best and worst case generalizations under the shift. We present
applications of our theory to real and synthetic datasets and for many kernels.
We compare results of our theory applied to Neural Tangent Kernel with
simulations of wide networks and show agreement. We analyze linear regression
in further depth.

24
The categories of this paper are: dg-ga
2
The categories of this paper are: math
37
The categories of this paper are: supr-con
22
The categories of this paper are: chao-dyn
11
The categories of this paper are: hep-lat
17
The categories of this paper are: q-fin
23
The categories of this paper are: solv-int
31
The categories of this paper are: ao-sci
14
The categories of this paper are: math-ph
26
The categories of this paper are: bayes-an
15
The categories of this paper are: nucl-ex
21
The categories of this paper are: adap-org
33
The categories of this paper are: plasm-ph
34
The categories of this paper are: chem-ph
36
The categories of this paper are: cmp-lg
10
The categories of this paper are: q-bio