Inference in Differences-in-Differences: How Much Should We Trust in Independent Clusters?

Ferman, Bruno (2019): Inference in Differences-in-Differences: How Much Should We Trust in Independent Clusters?

Abstract

We analyze the conditions in which ignoring spatial correlation is problematic for inference in differences-in-differences (DID) models. Assuming that the spatial correlation structure follows a linear factor model, we show that inference ignoring such correlation remains reliable when either (i) the second moment of the difference between the pre- and post-treatment averages of common factors is low, or (ii) the distribution of factor loadings has the same expected values for treated and control groups, and do not exhibit significant spatial correlation. We present simulations with real datasets that corroborate these conclusions. Our results provide important guidelines on how to minimize inference problems due to spatial correlation in DID applications.

• Item Type: MPRA Paper
• Language: English
• Keywords: inference, differences-in-differences, spatial correlation
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C12 - Hypothesis Testing: General; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C21 - Cross-Sectional Models ; Spatial Models ; Treatment Effect Models ; Quantile Regressions; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C23 - Panel Data Models ; Spatio-temporal Models; C - Mathematical and Quantitative Methods > C3 - Multiple or Simultaneous Equation Models ; Multiple Variables > C33 - Panel Data Models ; Spatio-temporal Models

References

Adao, R., Kolesar, M., and Morales, E. (2010). Shift-share designs: Theory and inference.

Arellano, M. (1987). Computing robust standard errors for within-groups estimators. Oxford Bulletin of Economics and Statistics, 49(4):431–434.

Athey, S. and Imbens, G. (2018). Design-based Analysis in Difference-In-Differences Settings with Staggered Adoption. Working Paper, arXiv:1808.05293

Barrios, T., Diamond, R., Imbens, G. W., and Kolesar, M. (2012). Clustering, spatial correlations, and randomization inference. Journal of the American Statistical Association, 107(498):578–591.

Bertrand, M., Duflo, E., and Mullainathan, S. (2004). How much should we trust diff in-diff estimates? Quarterly Journal of Economics, page 24975.

Bester, C. A., Conley, T. G., and Hansen, C. B. (2011). Inference with dependent data using cluster covariance estimators. Journal of Econometrics, 165(2):137 – 151.

Brewer, M., Crossley, T. F., and Joyce, R. (2013). Inference with Difference-in-Differences Revisited. IZA Discussion Papers 7742, Institute for the Study of Labor (IZA).

Callaway, B. and Sant'Anna, P. H. C. (2018). Difference-in-Differences with Multiple Time Periods and an Application on the Minimum Wage and Employment. Working Paper,arXiv:1803.09015 .

Cameron, A., Gelbach, J., and Miller, D. (2008). Bootstrap-based improvements for inference with clustered errors. The Review of Economics and Statistics, 90(3):414–427.

Cameron, A. C., Gelbach, J. B., and Miller, D. L. (2011). Robust inference with multiway clustering. Journal of Business & Economic Statistics, 29(2):238–249.

Canay, I. A., Romano, J. P., and Shaikh, A. M. (2017). Randomization tests under an approximate symmetry assumption. Econometrica, 85(3):1013–1030.

Conley, T. G. and Taber, C. R. (2011). Inference with Difference in Differences with a Small Number of Policy Changes. The Review of Economics and Statistics, 93(1):113–125.

Dailey, A. (2017). Randomization inference with rainfall data: Using historical weather patterns for variance estimation. Political Analysis, 25(3):277 – 288.

Davezies, L., D'Haultfoeuille, X., and Guyonvarch, Y. (2018). Asymptotic results under multiway clustering. arXiv e-prints, page arXiv:1807.07925.

Ferman, B. and Pinto, C. (2019). Inference in differences-in-diff with few treated groups and heteroskedasticity. The Review of Economics and Statistics, 0(ja):null.

Goodman-Bacon, A. (2018). Difference-in-differences with variation in treatment timing. Working Paper 25018, National Bureau of Economic Research.

Kim, M. S. and Sun, Y. (2013). Heteroskedasticity and spatiotemporal dependence robust inference for linear panel models with fi effects. Journal of Econometrics, 177(1):85 – 108.

MacKinnon, J. G., Nielsen, M., and Webb, M. D. (2019). Wild Bootstrap and Asymptotic Inference with Multiway Clustering. Working Paper 1415, Economics Department, Queen's University.

MacKinnon, J. G. and Webb, M. D. (2015). Differences-in-Differences Inference with Few Treated Clusters.

MacKinnon, J. G. and Webb, M. D. (2017). Wild bootstrap inference for wildly diff t cluster sizes. Journal of Applied Econometrics, 32(2):233–254.

Menzel, K. (2017). Bootstrap with Clustering in Two or More Dimensions. arXiv e-prints, page arXiv:1703.03043.

Ruggles, S., Genadek, K., Goeken, R., Grover, J., and Sobek, M. (2015). Integrated Public Use Microdata Series: Version 6.0 [Machine-readable database].

Thompson, S. B. (2011). Simple formulas for standard errors that cluster by both firm and time. Journal of Financial Economics, 99(1):1-10.

Vogelsang, T. J. (2012). Heteroskedasticity, autocorrelation, and spatial correlation robust inference in linear panel models with fixed-eff Journal of Econometrics, 166(2):303 – 319.