From Replications to Revelations: Heteroskedasticity-Robust Inference

Kranz, Sebastian (2024): From Replications to Revelations: Heteroskedasticity-Robust Inference.

Abstract

We compare heteroskedasticity-robust inference methods with a large-scale Monte Carlo study based on regressions from 155 reproduction packages of leading economic journals. The results confirm established wisdom and uncover new insights. Among well established methods HC2 standard errors with the degree of freedom specification proposed by Bell and McCaffrey (2002) perform best. To further improve the accuracy of t-tests, we propose a novel degree-of-freedom specification based on partial leverages. We also show how HC2 to HC4 standard errors can be refined by more effectively addressing the 15.6% of cases where at least one observation exhibits a leverage of one.

• Item Type: MPRA Paper
• Language: English
• Keywords: hetereoskedasticity, robust standard errors, meta study, replications, degree of freedom correction
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C12 - Hypothesis Testing: General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C15 - Statistical Simulation Methods: General; C - Mathematical and Quantitative Methods > C8 - Data Collection and Data Estimation Methodology ; Computer Programs > C87 - Econometric Software

References

• Athey, S., Tibshirani, J., & Wager, S. (2019). “Generalized random forests”. The Annals of Statistics, 47(2), 1148.

• Bell, R. M., & McCaffrey, D. F. (2002). “Bias reduction in standard errors for linear regression with multi-stage samples”. Survey Methodology, 28(2), 169-182.

• Berge L (2018). “Efficient estimation of maximum likelihood models with multiple fixed-effects: the R package FENmlm”. CREA Discussion Papers.

• Cattaneo, M. D., M. Jansson, and W. K. Newey. 2018. “Inference in linear regression models with many covariates and heteroscedasticity”. Journal of the American Statistical Association 113: 1350–1361.

• Chesher, A., and I. Jewitt. 1987. “The bias of a heteroskedasticity consistent covariance matrix estimator”. Econometrica 55: 1217–1222.

• Chesher, A., and G. Austin. 1991. “The finite-sample distributions of heteroskedasticity robust Wald statistics”. Journal of Econometrics 47: 153–173.

• Correia, Sergio. 2017. “Linear Models with High-Dimensional Fixed Effects: An Efficient and Feasible Estimator”. Working Paper. http://scorreia.com/research/hdfe.pdf

• Cribari-Neto, F. (2004). “Asymptotic inference under heteroskedasticity of unknown form”. Computational Statistics & Data Analysis, 45(2), 215-233.

• Christensen, Rune Haubo B (2018), “Satterthwaite’s Method for Degrees of Freedom in Linear Mixed Models”. Notes for the R package lmerTestR

• Davidson, R., & MacKinnon, J. G. (1993). “Econometric Theory and Methods”. Oxford University Press

• Ding, P. (2021). “The Frisch–Waugh–Lovell theorem for standard errors”. Statistics & Probability Letters, 168, 108945.

• Eicker, Friedhelm (1967). “Limit Theorems for Regression with Unequal and Dependent Errors”. Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability. Vol. 5. pp. 59–82.

• Hansen, B. (2022). “Econometrics”. Princeton University Press.

• Huber, Peter J. (1967). "The behavior of maximum likelihood estimates under nonstandard conditions". Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability. Vol. 5. pp. 221–233.

• Imbens, G. W., & Kolesár, M. (2016). “Robust standard errors in small samples: Some practical advice”. Review of Economics and Statistics, 98(4), 701-712.

• Kolesár, M. (2023), “Robust Standard Errors in Small Samples”. Vignette of the R package dfadjust.

• Pötscher, B. M., & Preinerstorfer, D. (2023). “Valid Heteroskedasticity Robust Testing”. Econometric Theory, 1–53.

• Pustejovsky, J. E., & Tipton, E. (2017). “Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models”. Journal of Business & Economic Statistics, 36(4), 672–683.

• MacKinnon, James G.; White, Halbert (1985). “Some Heteroskedastic-Consistent Covariance Matrix Estimators with Improved Finite Sample Properties”. Journal of Econometrics. 29 (3): 305–325.

• MacKinnon, J. G., Nielsen, M. Ø., & Webb, M. D. (2023a). “Fast and reliable jackknife and bootstrap methods for cluster‐robust inference”. Journal of Applied Econometrics, 38(5), 671-694.

• MacKinnon, J. G., Nielsen, M. Ø., & Webb, M. D. (2023b). “Leverage, influence, and the jackknife in clustered regression models: Reliable inference using summclust”. The Stata Journal, 23(4), 942-982.

• Roodman, D., Nielsen, M. Ø., MacKinnon, J. G., & Webb, M. D. (2019). “Fast and wild: Bootstrap inference in Stata using boottest”. The Stata Journal, 19(1), 4-60.

• Satterthwaite, F. E. (1946), “An Approximate Distribution of Estimates of Variance Components,” Biometrics Bulletin 2, 110–114.

• White, Halbert (1980). “A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity”. Econometrica. 48 (4): 817–838.

• Wu, C. F. J. (1986). “Jackknife, bootstrap and other resampling methods in regression analysis”. the Annals of Statistics, 14(4), 1261-1295.

• Young, A. (2019). “Channeling fisher: Randomization tests and the statistical insignificance of seemingly significant experimental results”. The Quarterly Journal of Economics, 134(2), 557-598.

• Young, A. (2022). “Consistency without inference: Instrumental variables in practical application”. European Economic Review, 147, 104-112.

• Zeileis, A. (2004). “Econometric Computing with HC and HAC Covariance Matrix Estimators”. Journal of Statistical Software, 11(10), 1–17.