TITLE: Non-sparse methods for out-of-sample prediction
in high-dimensional linear models

SPEAKER:  Dr. Lee Dicker

ABSTRACT:

Motivated by questions about dense (non-sparse) signals in
high-dimensional data analysis, we study the unconditional out-of-sample
prediction error (predictive risk) associated with three classes of dense
estimators for high-dimensional linear models: Ridge regression
estimators, scalar multiples of the ordinary least squares estimator
(which we refer to as James-Stein estimators), and marginal regression
estimators. Our results require no assumptions about sparsity and imply
that in linear models where the number of predictors is roughly
proportional to the number of observations:
(i) If the population predictor covariance is known (or if a
norm-consistent estimator is available), then the ridge estimator
outperforms the James-Stein estimator; (ii) both the ridge and James-Stein
estimators outperform the ordinary least squares estimator, and the
improvements offered by these estimators are especially significant when
the signal-to-noise ratio is small; and (iii) the marginal estimator has
serious deficiencies for out-of-sample prediction. We derive new
closed-form expressions for the asymptotic predictive risk of the
estimators, which allow us to precisely quantify the previous claims.
Additionally, minimax ridge and James-Stein estimators are identified.
Finally, we argue that the ridge estimator is, in fact, asymptotically
optimal among dense estimators for out-of-sample prediction in
high-dimensional linear models.

Contact: Lee Dicker <ldicker@stat.rutgers.edu>