Semiparametric penalty function method in partially linear model selection

Dong, Chaohua and Gao, Jiti and Tong, Howell (2006): Semiparametric penalty function method in partially linear model selection. Published in: Statistica Sinica , Vol. 17, No. 1 (October 2007): pp. 99-114.

Abstract

Model selection in nonparametric and semiparametric regression is of both theoretical and practical interest. Gao and Tong (2004) proposed a semiparametric leave–more–out cross–validation selection procedure for the choice of both the parametric and nonparametric regressors in a nonlinear time series regression model. As recognized by the authors, the implementation of the proposed procedure requires the availability of relatively large sample sizes. In order to address the model selection problem with small or medium sample sizes, we propose a model selection procedure for practical use. By extending the so–called penalty function method proposed in Zheng and Loh (1995, 1997) through the incorporation of features of the leave-one-out cross-validation approach, we develop a semiparametric, consistent selection procedure suitable for the choice of optimum subsets in a partially linear model. The newly proposed method is implemented using the full set of data, and simulations show that it works well for both small and medium sample sizes.

• Item Type: MPRA Paper
• Language: English
• Keywords: Linear model; model selection; nonparametric method; partially linear model; semiparametric method
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C14 - Semiparametric and Nonparametric Methods: General

References

Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In 2nd International Symposium on Information Theory (Edited by B. N. Petrov and F. Csaki), 267-281. Akademiai Kiado, Budapest.

Cheng, B. and Tong, H. (1992). On consistent nonparametric order determination and chaos. J. Roy. Statist. Soc. Ser. B 54, 427-449.

Gao, J. and Tong, H. (2002). Nonparametric and semiparametric regression model selection. Working paper.

Gao, J. and Tong, H. (2004). Semiparametric nonlinear time series model selection. J. Roy. Statist. Soc. Ser. B 66, 321-336.

Gao, J. and Yee, T. (2000). Adaptive estimation in partially linear (semiparametric) autoregressive models. Canad. J. Statist. 28, 571-586.

Hardle, W., Hall, P. and Marron, J. (2002). Regression smoothing parameters that are not far from their optimum. J. Amer. Statist. Assoc. 87, 227-233.

Mallows, C. L. (1973). Some comments on Cp. Technometrics 15, 661-675.

Masry, E. and Tjøstheim, D. (1995). Nonparametric estimation and identification of nonlinear ARCH time series. Econometric Theory 11, 258-289.

Masry, E. and Tjostheim, D. (1997). Additive nonlinear ARX time series and projection estimates. Econom. Theory 13, 214-252.

Shao, J. (1993). Linear model selection by cross–validation. J. Amer. Statist. Assoc. 422, 486494.

Shibata, R. (1976). Selection of the order of an autoregressive model by Akaike’s information criterion. Biometrika 63, 117-126.

Vieu, P. (1994). Choice of regressors in nonparametric estimation. Comput. Statist. Data Anal. 17, 575-594.

Yao, Q. and Tong, H. (1994). On subset selection in nonparametric stochastic regression. Statist. Sinica 4, 51-70.

Zhang, P. (1993). Model selection via multifold cross–validation. Ann. Statist. 21, 299-310.

Zheng, X. and Loh, W. Y. (1995). Consistent variable selection in linear models. J. Amer. Statist. Assoc. 90, 151-156.

Zheng, X. and Loh, W. Y. (1997). A consistent variable selection criterion for linear models with high–dimensional covariates. Statist. Sinica 7, 311-325.