A plug-in averaging estimator for regressions with heteroskedastic errors

LIU, CHU-AN (2012): A plug-in averaging estimator for regressions with heteroskedastic errors.

Abstract

This paper proposes a new model averaging estimator for the linear regression model with heteroskedastic errors. We address the issues of how to optimally assign the weights for candidate models and how to make inference based on the averaging estimator. We derive the asymptotic mean squared error (AMSE) of the averaging estimator in a local asymptotic framework, and then choose the optimal weights by minimizing the AMSE. We propose a plug-in estimator of the optimal weights and use these estimated weights to construct a plug-in averaging estimator of the parameter of interest. We derive the asymptotic distribution of the plug-in averaging estimator and suggest a plug-in method to construct confidence intervals. Monte Carlo simulations show that the plug-in averaging estimator has much smaller expected squared error, maximum risk, and maximum regret than other existing model selection and model averaging methods. As an empirical illustration, the proposed methodology is applied to cross-country growth regressions.

• Item Type: MPRA Paper
• Language: English
• Keywords: Local asymptotic theory, Model averaging, Model selection, Plug-in estimators
• Subjects: C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C51 - Model Construction and Estimation; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C52 - Model Evaluation, Validation, and Selection

References

Akaike, H. (1973): “Information Theory and an Extension of the Maximum Likelihood Principle,” in Second International Symposium on Information Theory, ed. by B. Petroc, and F. Csake, pp. 267–281. Akademiai Kiado.

Andrews, D. (1991a): “Asymptotic Optimality of Generalized CL, Cross-Validation, and Generalized Cross-Validation in Regression with Heteroskedastic Errors,” Journal of Econometrics, 47, 359–377.

Andrews, D. (1991b): “Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation,” Econometrica, 59, 817–858.

Buckland, S., K. Burnham, and N. Augustin (1997): “Model Selection: An Integral Part of Inference,” Biometrics, 53, 603–618.

Burnham, K., and D. Anderson (2002): Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach. Springer Verlag.

Claeskens, G., and N. Hjort (2008): Model Selection and Model Averaging. Cambridge University Press.

DiTraglia, F. (2011): “Using Invalid Instruments on Purpose: Focused Moment Selection and Averaging for GMM,” Unpublished Manuscript.

Durlauf, S., A. Kourtellos, and C. Tan (2008): “Are Any Growth Theories Robust?,” The Economic Journal, 118(527), 329–346.

Durlauf, S. N., P. A. Johnson, and J. R. Temple (2005): “Growth Econometrics,” in Handbook of Economic Growth, ed. by P. Aghion, and S. Durlauf, vol. 1, pp. 555–677. Elsevier.

Fernandez, C., E. Ley, and M. Steel (2001): “Model Uncertainty in Cross-Country Growth Regressions,” Journal of Applied Econometrics, 16(5), 563–576.

Hansen, B. (2007): “Least Squares Model Averaging,” Econometrica, 75(4), 1175–1189.

Hansen, B. (2009): “Averaging Estimators for Regressions with a Possible Structural Break,” Econometric Theory, 25(06), 1498–1514.

Hansen, B. (2010): “Averaging Estimators for Autoregressions with a Near Unit Root,” Journal of Econometrics, 158(1), 142–155.

Hansen, B., and J. Racine (2012): “Jackknife Model Averaging,” Journal of Econometrics, 167(1), 38–46.

Hansen, P., A. Lunde, and J. Nason (2011): “The Model Confidence Set,” Econometrica, 79(2), 453–497.

Hausman, J. (1978): “Specification Tests in Econometrics,” Econometrica, 46, 1251–1271.

Hjort, N., and G. Claeskens (2003): “Frequentist Model Average Estimators,” Journal of the American Statistical Association, 98(464), 879–899.

Hoeting, J., D. Madigan, A. Raftery, and C. Volinsky (1999): “Bayesian Model Averaging: A Tutorial,” Statistical Science, 14, 382–401.

Kabaila, P. (1995): “The Effect of Model Selection on Confidence Regions and Prediction Regions,” Econometric Theory, 11, 537–537.

Kabaila, P. (1998): “Valid Confidence Intervals in Regression after Variable Selection,” Econometric Theory, 14(4), 463–482.

Kim, J., and D. Pollard (1990): “Cube Root Asymptotics,” The Annals of Statistics, 18, 191–219.

Leeb, H., and B. P¨otscher (2003): “The Finite-Sample Distribution of Post-Model-Selection Estimators and Uniform versus Non-Uniform Approximations,” Econometric Theory, 19(1), 100–142.

Leeb, H., and B. P¨otscher (2005): “Model Selection and Inference: Facts and Fiction,” Econometric Theory, 21(1), 21–59.

Leeb, H., and B. P¨otscher (2006): “Can One Estimate the Conditional Distribution of Post-Model-Selection Estimators?,” The Annals of Statistics, 34(5), 2554–2591.

Leeb, H., and B. P¨otscher (2008): “Can One Estimate the Unconditional Distribution of Post-Model-Selection Estimators?,” Econometric Theory, 24(02), 338–376.

Leung, G., and A. Barron (2006): “Information Theory and Mixing Least-Squares Regressions,” IEEE Transactions on Information Theory, 52(8), 3396–3410.

Li, K.-C. (1987): “Asymptotic Optimality for Cp, CL, Cross-Validation and Generalized Cross-Validation: Discrete Index Set,” The Annals of Statistics, 15, 958–975.

Liang, H., G. Zou, A. Wan, and X. Zhang (2011): “Optimal Weight Choice for Frequentist Model Average Estimators,” Journal of the American Statistical Association, 106, 1053–1066.

Magnus, J., O. Powell, and P. Prufer (2010): “A Comparison of Two Model Averaging Techniques with an Application to Growth Empirics,” Journal of Econometrics, 154(2), 139–153.

Newey, W., and K. West (1987): “A Simple, Positive Semi-Definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix,” Econometrica, 55(3), 703–708.

P¨otscher, B. (1991): “Effects of Model Selection on Inference,” Econometric Theory, 7(2), 163–185.

P¨otscher, B. (2006): “The Distribution of Model Averaging Estimators and an Impossibility Result Regarding its Estimation,” Lecture Notes-Monograph Series, 52, 113–129.

Sala-i Martin, X., G. Doppelhofer, and R. Miller (2004): “Determinants of Long-Term Growth: A Bayesian Averaging of Classical Estimates (BACE) Approach,” American Economic Review, 94, 813–835.

Staiger, D., and J. Stock (1997): “Instrumental Variables Regression with Weak Instruments,” Econometrica, 65, 557–586.

Tibshirani, R. (1996): “Regression Shrinkage and Selection via the Lasso,” Journal of the Royal Statistical Society. Series B (Methodological), 58, 267–288.

Van der Vaart, A., and J. Wellner (1996): Weak Convergence and Empirical Processes. Springer Verlag.

Wan, A., X. Zhang, and G. Zou (2010): “Least SquaresModel Averaging by Mallows Criterion,” Journal of Econometrics, 156(2), 277–283.

White, H. (1980): “A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity,” Econometrica, 48, 817–838.

White, H. (1984): Asymptotic Theory for Econometricians. Academic Press.

White, H., and X. Lu (2010): “Robustness Checks and Robustness Tests in Applied Economics,” Unpublished Manuscript.

Yang, Y. (2001): “Adaptive Regression by Mixing,” Journal of the American Statistical Association, 96(454), 574–588.

Yuan, Z., and Y. Yang (2005): “Combining Linear Regression Models: When and How?,” Journal of the American Statistical Association, 100, 1202–1214.

Zou, H. (2006): “The Adaptive Lasso and Its Oracle Properties,” Journal of the American Statistical Association, 101(476), 1418–1429.