Distribution Theory of the Least Squares Averaging Estimator

Liu, Chu-An (2013): Distribution Theory of the Least Squares Averaging Estimator.

Abstract

This paper derives the limiting distributions of least squares averaging estimators for linear regression models in a local asymptotic framework. We show that the averaging estimators with fixed weights are asymptotically normal and then develop a plug-in averaging estimator that minimizes the sample analog of the asymptotic mean squared error. We investigate the focused information criterion (Claeskens and Hjort, 2003), the plug-in averaging estimator, the Mallows model averaging estimator (Hansen, 2007), and the jackknife model averaging estimator (Hansen and Racine, 2012). We find that the asymptotic distributions of averaging estimators with data-dependent weights are nonstandard and cannot be approximated by simulation. To address this issue, we propose a simple procedure to construct valid confidence intervals with improved coverage probability. Monte Carlo simulations show that the plug-in averaging estimator generally has smaller expected squared error than other existing model averaging methods, and the coverage probability of proposed confidence intervals achieves the nominal level. As an empirical illustration, the proposed methodology is applied to cross-country growth regressions.

• Item Type: MPRA Paper
• Language: English
• Keywords: Local asymptotic theory, Model averaging, Model selection, Plug-in estimators.
• Subjects: C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C51 - Model Construction and Estimation; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C52 - Model Evaluation, Validation, and Selection

References

Andrews, D. W. K. (1991a): “Asymptotic Optimality of Generalized CL, Cross-Validation, and Generalized Cross-Validation in Regression with Heteroskedastic Errors,” Journal of Economet- rics, 47, 359–377.

——— (1991b): “Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation,” Econometrica, 59, 817–858.

Buckland, S., K. Burnham, and N. Augustin (1997): “Model Selection: An Integral Part of Inference,” Biometrics, 53, 603–618.

Claeskens, G. and R. J. Carroll (2007): “An Asymptotic Theory forModel Selection Inference in General Semiparametric Problems,” Biometrika, 94, 249–265.

Claeskens, G. and N. L. Hjort (2003): “The Focused Information Criterion,” Journal of the American Statistical Association, 98, 900–916.

——— (2008): Model Selection and Model Averaging, Cambridge University Press.

DiTraglia, F. (2013): “Using Invalid Instruments on Purpose: Focused Moment Selection and Averaging for GMM,” Working Paper, University of Pennsylvania.

Durlauf, S., A. Kourtellos, and C. Tan (2008): “Are Any Growth Theories Robust?” The Economic Journal, 118, 329–346.

Durlauf, S. N., P. A. Johnson, and J. R. Temple (2005): “Growth Econometrics,” in Handbook of Economic Growth, ed. by P. Aghion and S. Durlauf, Elsevier, vol. 1, 555–677.

Elliott, G., A. Gargano, and A. Timmermann (2013): “Complete Subset Regressions,” Journal of Econometrics, 177, 357–373.

Fernandez, C., E. Ley, and M. Steel (2001): “Model Uncertainty in Cross-Country Growth Regressions,” Journal of Applied Econometrics, 16, 563–576.

Hansen, B. E. (2007): “Least Squares Model Averaging,” Econometrica, 75, 1175–1189.

——— (2009): “Averaging Estimators for Regressions with a Possible Structural Break,” Econometric Theory, 25, 1498–1514.

——— (2010): “Averaging Estimators for Autoregressions with a Near Unit Root,” Journal of Econometrics, 158, 142–155.

——— (2013a): “Econometrics,” Unpublished Manuscript, University of Wisconsin.

——— (2013b): “Model Averaging, Asymptotic Risk, and Regressor Groups,” Forthcoming. Quantitative Economics.

Hansen, B. E. and J. Racine (2012): “Jackknife Model Averaging,” Journal of Econometrics, 167, 38–46.

Hansen, P., A. Lunde, and J. Nason (2011): “The Model Confidence Set,” Econometrica, 79, 453–497.

Hausman, J. (1978): “Specification Tests in Econometrics,” Econometrica, 46, 1251–1271.

Hjort, N. L. and G. Claeskens (2003a): “Frequentist Model Average Estimators,” Journal of the American Statistical Association, 98, 879–899.

——— (2003b): “Rejoinder to “The Focused Information Criterion” and “Frequentist Model Average Estimators”,” Journal of the American Statistical Association, 98, 938–945.

Hoeting, J., D. Madigan, A. Raftery, and C. Volinsky (1999): “Bayesian Model Averaging: A Tutorial,” Statistical Science, 14, 382–401.

Kabaila, P. (1995): “The Effect of Model Selection on Confidence Regions and Prediction Regions,” Econometric Theory, 11, 537–537.

——— (1998): “Valid Confidence Intervals in Regression after Variable Selection,” Econometric Theory, 14, 463–482.

Kim, J. and D. Pollard (1990): “Cube Root Asymptotics,” The Annals of Statistics, 18, 191– 219.

Leeb, H. and B. P¨otscher (2003): “The Finite-Sample Distribution of Post-Model-Selection Estimators and Uniform versus Non-Uniform Approximations,” Econometric Theory, 19, 100– 142.

———(2005): “Model Selection and Inference: Facts and Fiction,” Econometric Theory, 21, 21–59.

——— (2006): “Can One Estimate the Conditional Distribution of Post-Model-Selection Estimators?” The Annals of Statistics, 34, 2554–2591.

——— (2008): “Can One Estimate the Unconditional Distribution of Post-Model-Selection Estimators?” Econometric Theory, 24, 338–376.

——— (2012): “Testing in the Presence of Nuisance Parameters: Some Comments on Tests Post- Model-Selection and Random Critical Values,” Working Paper, University of Vienna.

Leung, G. and A. Barron (2006): “Information Theory and Mixing Least-Squares Regressions,” IEEE Transactions on Information Theory, 52, 3396–3410.

Li, K.-C. (1987): “Asymptotic Optimality for Cp, CL, Cross-Validation and Generalized Cross- Validation: Discrete Index Set,” The Annals of Statistics, 15, 958–975.

Liang, H., G. Zou, A. Wan, and X. Zhang (2011): “Optimal Weight Choice for Frequentist Model Average Estimators,” Journal of the American Statistical Association, 106, 1053–1066.

Magnus, J., O. Powell, and P. Prufer (2010): “A Comparison of Two Model Averaging Techniques with an Application to Growth Empirics,” Journal of Econometrics, 154, 139–153.

Moral-Benito, E. (2013): “Model Averaging in Economics: An Overview,” forthcoming Journal of Economic Surveys.

Newey, W. and K. West (1987): “A Simple, Positive Semi-Definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix,” Econometrica, 55, 703–708.

P¨otscher, B. (1991): “Effects of Model Selection on Inference,” Econometric Theory, 7, 163–185. ——— (2006): “The Distribution of Model Averaging Estimators and an Impossibility Result Regarding its Estimation,” Lecture Notes-Monograph Series, 52, 113–129.

Raftery, A. E. and Y. Zheng (2003): “Discussion: Performance of Bayesian Model Averaging,” Journal of the American Statistical Association, 98, 931–938.

Sala-i Martin, X., G. Doppelhofer, and R. Miller (2004): “Determinants of Long-Term Growth: A Bayesian Averaging of Classical Estimates (BACE) Approach,” American Economic Review, 94, 813–835.

Staiger, D. and J. Stock (1997): “Instrumental Variables Regression with Weak Instruments,” Econometrica, 65, 557–586.

Tibshirani, R. (1996): “Regression Shrinkage and Selection via the Lasso,” Journal of the Royal Statistical Society. Series B (Methodological), 58, 267–288.

Van der Vaart, A. and J. Wellner (1996): Weak Convergence and Empirical Processes, Springer Verlag.

Wan, A., X. Zhang, and G. Zou (2010): “Least SquaresModel Averaging byMallows Criterion,” Journal of Econometrics, 156, 277–283.

White, H. (1980): “A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity,” Econometrica, 48, 817–838.

——— (1984): Asymptotic Theory for Econometricians, Academic Press.

White, H. and X. Lu (2014): “Robustness Checks and Robustness Tests in Applied Economics,” Journal of Econometrics, 178, Part 1, 194 – 206.

Yang, Y. (2000): “Combining Different Procedures for Adaptive Regression,” Journal of Multi- variate Analysis, 74, 135–161.

——— (2001): “Adaptive Regression by Mixing,” Journal of the American Statistical Association, 96, 574–588.

Yuan, Z. and Y. Yang (2005): “Combining Linear Regression Models: When and How?” Journal of the American Statistical Association, 100, 1202–1214.

Zhang, X. and H. Liang (2011): “Focused Information Criterion and Model Averaging for Generalized Additive Partial Linear Models,” The Annals of Statistics, 39, 174–200.

Zou, H. (2006): “The Adaptive Lasso and Its Oracle Properties,” Journal of the American Statis- tical Association, 101, 1418–1429.