Heteroskedasticity and Clustered Covariances from a Bayesian Perspective

Lewis, Gabriel (2022): Heteroskedasticity and Clustered Covariances from a Bayesian Perspective.

Abstract

We show that root-n-consistent heteroskedasticity-robust and cluster-robust regression estimators and confidence intervals can be derived from fully Bayesian models of population sampling. In our model, the vexed question of how and when to “cluster” is answered by the sampling design encoded in the model: simple random sampling implies a heteroskedasticity-robust Bayesian estimator, and clustered sampling implies a cluster-robust Bayesian estimator, providing a Bayesian parallel to the work of Abadie et al. (2017). Our model is based on the Finite Dirichlet Process (FDP), a well-studied population sampling process that apparently originates with R.A. Fisher, and our findings may not be surprising to readers familiar with the frequentist properties of the closely related Bayesian Bootstrap, Dirichlet Process, and Efron “pairs” or “block” bootstraps. However, our application of FDP to robust regression is novel, and it fills a gap concerning Bayesian cluster-robust regression. Our approach has several advantages over related methods: we present a full probability model with clear assumptions about a sampling design, one that does not assume that all possible data-values have been observed (unlike many bootstrap procedures); and our posterior estimates and credible intervals can be regularized toward reasonable prior values in small samples, while achieving the desirable frequency properties of a bootstrap in moderate and large samples. However, our approach also illustrates some limitations of “robust” procedures.

• Item Type: MPRA Paper
• Language: English
• Keywords: Bayesian; Heteroskedastic; Clustered Covariance; Robust Covariance; Sandwich Estimator
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C11 - Bayesian Analysis: General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C14 - Semiparametric and Nonparametric Methods: General; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling; C - Mathematical and Quantitative Methods > C8 - Data Collection and Data Estimation Methodology ; Computer Programs > C83 - Survey Methods ; Sampling Methods

References

Abadie, A., Athey, S., Imbens, G., & Wooldridge, J. (2017). When Should You Adjust Standard Errors for Clustering? arXiv:1710.02926 [econ, math, stat]. arXiv: 1710.02926 [econ, math, stat]

Abadie, A., Athey, S., Imbens, G., & Wooldridge, J. (2020). Sampling-Based versus Design-Based Uncertainty in Regression Analysis. Econometrica, 88(1), 265–296. doi:10.3982/ECTA12675

Aitkin, M. (2008). Applications of the Bayesian Bootstrap in Finite Population Inference. Journal of Official Statistics, 24(1), 21–51.

Bertrand, M., Duflo, E., & Mullainathan, S. (2004). How Much Should We Trust Differences-in- Differences Estimates? Quarterly Journal of Economics, 27.

Buja, A., Berk, R., Brown, L., George, E., Pitkin, E., Traskin, M., . . . Zhang, K. (2019). Models as Approximations I: Consequences Illustrated with Linear Regression. arXiv:1404.1578 [stat]. arXiv: 1404.1578 [stat]

Cameron, A. C., & Miller, D. L. (2015). A Practitioner’s Guide to Cluster-Robust Inference. Journal of Human Resources, 50(2), 317–372. doi:10.3368/jhr.50.2.317

Carter, A. V., Schnepel, K. T., & Steigerwald, D. G. (2017). Asymptotic Behavior of a t -Test Robust to Cluster Heterogeneity. The Review of Economics and Statistics, 99(4), 698–709. doi:10.1162/REST_a_00639

Chamberlain, G., & Imbens, G. (2003). Nonparametric Applications of Bayesian Inference. Journal of Business & Economic Statistics, 21(1), 12–18. doi:10.1198/073500102288618711

Efron, B. (1979). Bootstrap Methods: Another Look at the Jackknife. The Annals of Statistics, 7(1), 1–26. doi:10.1007/978-1-4612-4380-9_41

Eicker, F. (1967). Limit Theorems for Regressions with Unequal and Dependent Errors. In Pro- ceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability (Vol. 1, p. 24). Berkeley: University of California Press.

MacKinnon, J., & Webb, M. (2019). When and how to deal with clustered errors in regression models. Queen’s Economics Department Working Paper, 1421.

Meng, X.-L., & Xie, X. (2014). I Got More Data, My Model is More Refined, but My Estimator is Getting Worse! Am I Just Dumb? Econometric Reviews, 33(1-4), 218–250. doi:10.1080/ 07474938.2013.808567

Muliere, P., & Secchi, P. (2003). Weak Convergence of a Dirichlet-Multinomial Process. gmj, 10 (2), 319–324. doi:10.1515/GMJ.2003.319

Norets, A. (2015). Bayesian regression with nonparametric heteroskedasticity. Journal of Econo- metrics, 185(2), 409–419. doi:10.1016/j.jeconom.2014.12.006

Pelenis, J. (2012). Bayesian Semiparametric Regression (tech. rep. No. 285). Institut für Höhere Studien. Wien.

Pelenis, J. (2014). Bayesian regression with heteroscedastic error density and parametric mean function. Journal of Econometrics, 178, 624–638. doi:10.1016/j.jeconom.2013.10.006

Pitman, J. (1996). Some developments of the Blackwell-MacQueen urn scheme. In Institute of Mathematical Statistics Lecture Notes - Monograph Series (pp. 245–267). doi:10.1214/lnms/ 1215453576

Poirier, D. J. (2011). Bayesian Interpretations of Heteroskedastic Consistent Covariance Estimators Using the Informed Bayesian Bootstrap. Econometric Reviews, 30(4), 457–468. doi:10.1080/ 07474938.2011.553542

Polpo, A., Louzada, F., Rifo, L. L. R., Stern, J. M., & Lauretto, M. (Eds.). (2015). Interdisciplinary Bayesian Statistics: EBEB 2014. doi:10.1007/978-3-319-12454-4

Ramsey, F. (1926). Truth and Probability. In The Foundations of Mathematics and other Logical Essays (p.156–198). New York: Harcourt, Brace and Company.

Rice, K., Lumley, T., & Szpiro, A. (2008). Trading bias for precision: Decision theory for intervals and sets. UW Biostatistics Working Paper Series, (336), 29.

Robert, C. P., Chopin, N., & Rousseau, J. (2009). Harold Jeffreys’s Theory of Probability Revisited. Statistical Science, 24(2). doi:10.1214/09-STS284

Rubin, D. (1981). The Bayesian Bootstrap. The Annals of Statistics, 9(1), 130–134. Shalizi, C. R. (2009). Dynamics of Bayesian updating with dependent data and misspecified models.

Electronic Journal of Statistics, 3(none). doi:10.1214/09-EJS485

Sims, C. (2010). Understanding Non-Bayesians.

Stan. (2022). Stan Development Team.

Startz, R. (2012). Bayesian Heteroskedasticity-Robust Standard Errors. UC Santa Barbara Working Papers.

Szpiro, A. A., Rice, K. M., & Lumley, T. (2010). Model-robust regression and a Bayesian “sandwich” estimator. The Annals of Applied Statistics, 4(4). doi:10.1214/10-AOAS362. arXiv: 1101.1402

van der Vaart, A. (1998). Asymptotic Statistics (First). New York: Cambridge University Press.

West, M. (1984). Outlier Models and Priors in Bayesian Linear Regression. Journal of the Royal Statistical Society, 46(3), 431–439.

White, H. (1980a). A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity. Econometrica, 48(4), 817–838.

White, H. (1980b). Using Least Squares to Approximate Unknown Regression Functions. International Economic Review, 21(1), 149. doi:10.2307/2526245

Wood, S. (2017). Ch2: Linear Mixed Models. In Generalized Additive Models (Second). Chapman and Hall/CRC.

Yin, G. (2009). Bayesian generalized method of moments. Bayesian Analysis, 4 (2). doi:10.1214/09- BA407

Zhang, X. (2008). A Very Gentle Note on the Construction of Dirichlet Process. Australian National University.

Zhao, Y. (2015). Bayesian Linear Regression with Conditional Heteroskedasticity. UC3M Working Papers, 24.