Cross-Validating Synthetic Controls

Becker, Martin and Klößner, Stefan and Pfeifer, Gregor (2017): Cross-Validating Synthetic Controls.

Abstract

While the literature on synthetic control methods mostly abstracts from out-of-sample measures, Abadie et al. (2015) have recently introduced a cross-validation approach. This technique, however, is not well-defined since it hinges on predictor weights which are not uniquely defined. We fix this issue, proposing a new, well-defined cross-validation technique, which we apply to the original Abadie et al. (2015) data. Additionally, we discuss how this new technique can be used for comparing different specifications based on out-of-sample measures, avoiding the danger of cherry-picking.

• Item Type: MPRA Paper
• Language: English
• Keywords: Synthetic Control Methods; Cross-Validation; Specification Search.
• Subjects: C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C22 - Time-Series Models ; Dynamic Quantile Regressions ; Dynamic Treatment Effect Models ; Diffusion Processes; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C52 - Model Evaluation, Validation, and Selection

References

Abadie, A., Diamond, A., and Hainmueller, J. (2010). Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program. Journal of the American Statistical Association, 105(490):493-505.

Abadie, A., Diamond, A., and Hainmueller, J. (2015). Comparative Politics and the Synthetic Control Method. American Journal of Political Science, 59(2):495-510.

Abadie, A. and Gardeazabal, J. (2003). The Economic Costs of Conflict: A Case Study of the Basque Country. The American Economic Review, 93(1):113-132.

Athey, S. and Imbens, G. W. (2017). The state of applied econometrics: Causality and policy evaluation. Journal of Economic Perspectives, 31(2):3-32.

Becker, M. and Klößner, S. (2017a). Fast and Reliable Computation of Generalized Synthetic Controls. Econometrics and Statistics, pages n/a-n/a.

Becker, M. and Klößner, S. (2017b). MSCMT: Multivariate Synthetic Control Method Using Time Series. R package version 1.3.0.

Ferman, B., Pinto, C., and Possebom, V. (2017). Cherry picking with synthetic controls. Working Paper.

Gardeazabal, J. and Vega-Bayo, A. (2017). An Empirical Comparison Between the Synthetic Control Method and Hsiao et al.’s Panel Data Approach to Program Evaluation. Journal of Applied Econometrics, 32(5):983-1002.

Klößner, S., Kaul, A., Pfeifer, G., and Schieler, M. (2017). Comparative politics and the synthetic control method revisited: A note on Abadie et al. (2015). Swiss Journal of Economics and Statistics, pages n/a-n/a.

Klößner, S. and Pfeifer, G. (2017). Outside the box: Using synthetic control methods as a forecasting technique. Applied Economics Letters, pages n/a-n/a.

Montalvo, J. G. (2011). Voting after the bombings: A natural experiment on the effect of terrorist attacks on democratic elections. The Review of Economics and Statistics, 93(4):1146-1154.

R Core Team (2017). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna.