Nonparametric and semiparametric regression model selection

Gao, Jiti and Tong, Howell (2002): Nonparametric and semiparametric regression model selection.

Abstract

It is known that semiparametric time series regression is often used without checking its suitability and compactness. In theory, this may result in dealing with an unnecessarily complicated model. In practice, one may encounter the computational difficulty caused by the spareness of the data. This is partly because the curse of dimensionality problem may still arise from using a semiparametric time series regression model. This paper suggests that in order to provide more precise predictions we need to choose the most significant regressors for both the parametric and nonparametric time series components. We develop a novel cross-validation based model selection procedure for the choice of both the parametric and nonparametric time series components in semiparametric time series regression, and then establish some asymptotic properties of the proposed model selection procedure. In addition, we demonstrate how to implement the model selection procedure in practice through using both simulated and real examples. Our empirical studies show that the proposed cross-validation selection procedure works well numerically.

• Item Type: MPRA Paper
• Language: English
• Keywords: Linear model, model selection; mixing process; nonlinear time series; nonparametric regression; semiparametric regression; strictly stationary process; variable selection
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C14 - Semiparametric and Nonparametric Methods: General

References

{\sc Bickel, P. and Zhang, P.} (1992) Variable selection in nonparametric regression with categorical covariates. {\em J. Amer. Statist. Assoc.}, {\bf 87}, 90--97.

{\sc Boente, G. and Fraiman, R.} (1988). Consistency of a nonparametric estimate of a density function for dependent variables. {\em J. Multi. Anal.}, {\bf 25}, 90--99.

{\sc Chen, R., Liu, J. and Tsay, R.} (1995) Additivity tests for nonlinear autoregression. {\em Biometrika}, {\bf 82}, 369--383.

{\sc Chen, R. and Tsay, R.} (1993) Nonlinear additive ARX models. {\em J. Amer. Statist. Assoc.}, {\bf 88}, 955--967.

{\sc Cheng, B. and Tong, H.} (1992) On consistent nonparametric order determination and chaos. {\em J. Roy. Statist. Soc. Ser. B} {\bf 54} 427--449.

{\sc Cheng, B. and Tong, H.} (1993) Nonparametric function estimation in noisy chaos. {\em Developments in Time Series Analysis} (ed. T. Subba Rao), 183--206. Chapman and Hall, London.

{\sc Fan, J.} (1992) Design-adaptive nonparametric regression. {\em J. Amer. Statist. Assoc.}, {\bf 87}, 998--1004.

{\sc Fan, J. and Gijbels, I.} (1996) {\em Local Polynomial Modelling and Its Applications}. Chapman and Hall, London.

{\sc Gao, J.} (1998) Semiparametric regression modelling of nonlinear time series. {\em Scand. J. Statist.}, {\bf 25}, 521--539.

{\sc Gao, J. and Anh, V.} (2000) A central limit theorem for a random quadratic form of strictly stationary processes. {\em Statist. \& Probab. Lett.}, {\bf 49}, 69--79.

{\sc Gao, J. and Liang, H.} (1995) Asymptotic normality of pseudo-LS estimator for partially linear autoregressive models. {\em Statist. Probab. Lett.}, {\bf 23}, 27--34.

{\sc Gao, J. and Tong, H.} (2004) Semiparametric nonlinear time series model selection. {\em J. Roy. Statist. Soc. Ser. B}, {\bf 66}, 321--336.

{\sc Gao, J., Tong, H. and Wolff, R.} (2002a) Adaptive series estimation in additive stochastic regression models. {\em Statistica Sinica}, {\bf 12}, 409--428.

{\sc Gao, J., Tong, H. and Wolff, R.} (2002b) Model specification tests in nonparametric stochastic regression models. {\em J. Multivariate. Anal.} {\bf 83}, 324--359.

{\sc Gao, J., Wolff, R., Anh, V.} (2001) Semiparametric methods in multivariate approximation problems. {\em J. Complexity}, {\bf 17}, 754--772.

{\sc Gao, J. and Yee, T.} (2000) Adaptive estimation in partially linear (semiparametric) autoregressive models. {\em Canad. J. Statist.}, {\bf 28}, 571--586.

{\sc H\"{a}rdle, W., Hall, P. and Marron, J.} (1988) How far are automatically chosen regression smoothing parameters from their optimum (with discussion) ? {\em J. Amer. Statist. Assoc.}, {\bf 83}, 86--99.

{\sc H\"{a}rdle, W., Liang, H. and Gao, J.} (2000) {\em Partially Linear Models}. Springer Series in Contributions to Statistics. Physica--Verlag, New York.

{\sc H\"{a}rdle, W., L\"{u}tkepohl, H. and Chen, R.} (1997) A review of nonparametric time series analysis. {\em International Statist. Rev.}, {\bf 65}, 49--72.

{\sc H\"{a}rdle, W. and Vieu, P.} (1992) Kernel regression smoothing of time series. {\em J. Time Ser. Anal.}, {\bf 13}, 209--232.

{\sc Masry, E. and Tj{\o}stheim, D.} (1995) Nonparametric estimation and identification of nonlinear ARCH time series. {\em Econometric Theory} {\bf 11}, 258--289.

{\sc Masry, E. and Tj{\o}stheim, D.} (1997) Additive nonlinear ARX time series and projection estimates. {\em Econometric Theory}, {\bf 13}, 214--252.

{\sc Robinson, P.} (1983) Nonparametric estimation for time series models. {\em J. Time Ser. Anal.}, {\bf 4}, 185--208.

{\sc Robinson, P.} (1988) Root-N-consistent semiparametric regression. {\em Econometrica}, {\bf 56}, 931--964.

{\sc Shao, J.} (1993) Linear model selection by cross--validation. {\em J. Amer. Statist. Assoc.}, {\bf 422}, 486--494.

{\sc Shao, J.} (1997) An asymptotic theory for linear model selection (with comments). {\em Statistica Sinica}, {\bf 7}, 221--264.

{\sc Stone, M.} (1977) An asymptotic equivalence of choice of model by cross-validation and Akaike's criterion. {\em J. Roy. Statist. Soc. Ser. B}, {\bf 39}, 44--47.

{\sc Ter\"{a}svirta, T., Tj{\o}stheim, D. and Granger, C. W. J.} (1994) Aspects of modelling nonlinear time series, in R. F. Engle and D. L. McFadden (eds), {\em Handbook of Econometrics}, {\bf 4}, 2919--2957.

{\sc Tj{\o}stheim, D.} (1994) Nonlinear time series: a selective review. {\em Scand. J. Statist.}, {\bf 21}, 97--130.

{\sc Tj{\o}stheim, D.} (1999) Nonparametric specification procedures for time series. {\em Asymptotics, nonparametrics, and time series}, {\bf 158}, 149--199. Statistics: Textbooks and Monographs. Dekker, New York.

{\sc Tj{\o}stheim, D. and Auestad, B.} (1994a) Nonparametric identification of nonlinear time series: projections. {\em J. Amer. Statist. Assoc.}, {\bf 89}, 1398--1409.

{\sc Tj{\o}stheim, D. and Auestad, B.} (1994b) Nonparametric identification of nonlinear time series: selecting significant lags. {\em J. Amer. Statist. Assoc.}, {\bf 89}, 1410--1419.

{\sc Tong, H.} (1976) Fitting a smooth moving average to noisy data. {\em IEEE Trans. Inf. Theory}, {\bf IT-26}, 493--496.

{\sc Tong, H.} (1990) {\em Nonlinear Time Series}. Oxford University Press, Oxford.

{\sc Vieu, P.} (1994) Choice of regressors in nonparametric estimation. {\em Computat. Statist. & Data Anal.}, {\bf 17}, 575--594.

{\sc Xia, Y., Tong, H., Li, W. K. and Zhu, L.} (2002) An adaptive estimation of dimension reduction space. {\em J. Roy. Statist. Soc. Ser. B}, {\bf 64} (in press).

{\sc Yao, Q. and Tong, H.} (1994) On subset selection in nonparametric stochastic regression. {\em Statistica Sinica}, {\bf 4}, 51--70.

{\sc Zhang, P.} (1991) Variable selection in nonparametric regression with continuous covariates. {\em Ann. Statist.}, {\bf 19}, 1869--1882.

{\sc Zhang, P.} (1993) Model selection via multifold cross--validation. {\em Ann. Statist.}, {\bf 21}, 299--313.

{\sc Zheng, X. and Loh, W. Y.} (1997) A consistent variable selection criterion for linear models with high--dimensional covariates. {\em Statistica Sinica}, {\bf 7}, 311--326.