Risk bounds are derived for regression estimation based on model selection over a unrestricted number of models. While a large list of models provides more flexibility, significant selection bias may occur with bias-correction based model selection criteria like AIC. We incorporate a model complexity penalty term in AIC to handle the selection bias. Resulting estimators are shown to achieve a trade-off among approximation error, estimation error and model complexity automatically without prior knowledge about the true regression function. As applications, we demonstrate adaptation property of these estimators over full and sparse approximation function classes with different smoothness. For high-dimensional function estimation by tensor product splines, we show with number of knots and spline order adaptively selected, least squares estimator converges at anticipated rates simultaneously for Sobolev classes with different interaction orders and smoothness parameters.
 

Copies of preprints are available from the author upon request.  Use the preprint number (located at the top of the page) and make the request directly to the author, Iowa State University, Department of Statistics, Snedecor Hall, Ames, IA 50011-1210.