Testing Complete - Functional Design of Linear Regression
Testing Complete - Functional Design of Linear Regression

An Analysis of Random Design Linear Regression
Daniel Hsu
1,2
, Sham M. Kakade
2
, and Tong Zhang
1
1
Department of Statistics, Rutgers University
2
Department of Statistics, Wharton School, University of Pennsylvania
"The random design setting for linear regression concerns estimators based on a random sam-
ple  of  covariate/response  pairs.   This  work  gives  explicit  bounds  on  the  prediction  error  for
the ordinary least squares estimator and the ridge regression estimator under mild assumptions
on the covariate/response distributions.  In particular, this work provides sharp results on the
\out-of-sample" prediction error, as opposed to the \in-sample" ( xed design) error.  Our anal-
ysis also explicitly reveals the e ect of noise vs. modeling errors.  The approach reveals a close
connection to the more traditional  xed design setting, and our methods make use of recent ad-
vances in concentration inequalities (for vectors and matrices).  We also describe an application
of our results to fast least squares computations."

SimplifyData/Codality