Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Stability Based Generalization Bounds for Exponential Family Langevin
  Dynamics   We study generalization bounds for noisy stochastic mini-batch iterative
algorithms based on the notion of stability. Recent years have seen key
advances in data-dependent generalization bounds for noisy iterative learning
algorithms such as stochastic gradient Langevin dynamics (SGLD) based on
stability (Mou et al., 2018; Li et al., 2020) and information theoretic
approaches (Xu and Raginsky, 2017; Negrea et al., 2019; Steinke and
Zakynthinou, 2020; Haghifam et al., 2020). In this paper, we unify and
substantially generalize stability based generalization bounds and make three
technical advances. First, we bound the generalization error of general noisy
stochastic iterative algorithms (not necessarily gradient descent) in terms of
expected (not uniform) stability. The expected stability can in turn be bounded
by a Le Cam Style Divergence. Such bounds have a O(1/n) sample dependence
unlike many existing bounds with O(1/\sqrt{n}) dependence. Second, we introduce
Exponential Family Langevin Dynamics(EFLD) which is a substantial
generalization of SGLD and which allows exponential family noise to be used
with stochastic gradient descent (SGD). We establish data-dependent expected
stability based generalization bounds for general EFLD algorithms. Third, we
consider an important special case of EFLD: noisy sign-SGD, which extends
sign-SGD using Bernoulli noise over {-1,+1}. Generalization bounds for noisy
sign-SGD are implied by that of EFLD and we also establish optimization
guarantees for the algorithm. Further, we present empirical results on
benchmark datasets to illustrate that our bounds are non-vacuous and
quantitatively much sharper than existing bounds.

9
The categories of this paper are: nlin
33
The categories of this paper are: plasm-ph
5
The categories of this paper are: gr-qc
11
The categories of this paper are: hep-lat
15
The categories of this paper are: nucl-ex
19
The categories of this paper are: eess
36
The categories of this paper are: cmp-lg
6
The categories of this paper are: astro-ph
23
The categories of this paper are: solv-int
35
The categories of this paper are: mtrl-th
10
The categories of this paper are: q-bio
13
The categories of this paper are: quant-ph
17
The categories of this paper are: q-fin
27
The categories of this paper are: comp-gas
14
The categories of this paper are: math-ph
0
The categories of this paper are: hep-ph