Synthetic Control Methods: Never Use All Pre-Intervention Outcomes Together With Covariates

Kaul, Ashok and Klößner, Stefan and Pfeifer, Gregor and Schieler, Manuel (2015): Synthetic Control Methods: Never Use All Pre-Intervention Outcomes Together With Covariates.

Abstract

It is becoming increasingly popular in applications of synthetic control methods to include the entire pre-treatment path of the outcome variable as economic predictors. We demonstrate both theoretically and empirically that using all outcome lags as separate predictors renders all other covariates irrelevant. This finding holds irrespective of how important these covariates are for accurately predicting post-treatment values of the outcome, potentially threatening the estimator's unbiasedness. We show that estimation results and corresponding policy conclusions can change considerably when the usage of outcome lags as predictors is restricted, resulting in other covariates obtaining positive weights.

• Item Type: MPRA Paper
• Language: English
• Keywords: Synthetic Control Methods; Economic Predictors; Counterfactuals; Policy Evaluation.
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C23 - Panel Data Models ; Spatio-temporal Models

References

Abadie, A., Diamond, A., and Hainmueller, J. (2010). Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program. Journal of the American Statistical Association, 105(490):493-505.

Abadie, A., Diamond, A., and Hainmueller, J. (2011). Synth: Stata module to implement synthetic control methods for comparative case studies. Statistical Software Components, Boston College Department of Economics.

Abadie, A., Diamond, A., and Hainmueller, J. (2015). Comparative politics and the synthetic control method. American Journal of Political Science, 59(2):495-510.

Abadie, A. and Gardeazabal, J. (2003). The economic costs of conflict: A case study of the Basque country. The American Economic Review, 93(1):113-132.

Acemoglu, D., Johnson, S., Kermani, A., Kwak, J., and Mitton, T. (2016). The value of connections in turbulent times: Evidence from the United States. Journal of Financial Economics, 121(2):368-391.

Athey, S. and Imbens, G. W. (2017). The state of applied econometrics: Causality and policy evaluation. Journal of Economic Perspectives, 31(2):3-32.

Bauhoff, S. (2014). The effect of school district nutrition policies on dietary intake and overweight: A synthetic control approach. Economics and Human Biology, 12:45-55.

Belot, M. and Vandenberghe, V. (2014). Evaluating the “threat” effects of grade repetition: Exploiting the 2001 reform by the french-speaking community of Belgium. Education Economics, 22(1/2):73-89.

Bilgel, F. and Galle, B. (2015). Financial incentives for kidney donation: A comparative case study using synthetic controls. Journal of Health Economics, 43:103-117.

Billmeier, A. and Nannicini, T. (2013). Assessing economic liberalization episodes: A synthetic control approach. The Review of Economics and Statistics, 95(3):983-1001.

Bohn, S., Lofstrom, M., and Raphael, S. (2014). Did the 2007 Legal Arizona Workers Act reduce the state’s unauthorized immigrant population? The Review of Economics and Statistics, 96(2):258-269.

Cavallo, E., Galiani, S., Noy, I., and Pantano, J. (2013). Catastrophic Natural Disasters and Economic Growth. The Review of Economics and Statistics, 95(5):1549-1561.

Coffman, M. and Noy, I. (2012). Hurricane Iniki: measuring the long-term economic impact of a natural disaster using synthetic control. Environment and Development Economics, 17(2):187-205.

Doudchenko, N. and Imbens, G. W. (2016). Balancing, regression, difference-in-differences and synthetic control methods: A synthesis. NBER Working Paper No. 22791.

Eren, O. and Ozbeklik, S. (2016). What do right-to-work laws do? Evidence from a synthetic control method analysis. Journal of Policy Analysis and Management, 35(1):173-194.

Ferman, B. and Pinto, C. (2017). Revisiting the synthetic control estimator. Working Paper.

Ferman, B., Pinto, C., and Possebom, V. (2017). Cherry picking with synthetic controls. Working Paper.

Gardeazabal, J. and Vega-Bayo, A. (2017). An empirical comparison between the synthetic control method and Hsiao et al.’s panel data approach to program evaluation. Journal of Applied Econometrics, pages n/a–n/a.

Gobillon, L. and Magnac, T. (2016). Regional policy evaluation: Interactive fixed effects and synthetic controls. The Review of Economics and Statistics, 98(3):535-551.

Hinrichs, P. (2012). The effects of affirmative action bans on college enrollment, educational attainment, and the demographic composition of universities. The Review of Economics and Statistics, 94(3):712-722.

Hosny, A. S. (2012). Algeria’s trade with GAFTA countries: A synthetic control approach. Transition Studies Review, 19(1):35-42.

Jinjarak, Y., Noy, I., and Zheng, H. (2013). Capital controls in Brazil – stemming a tide with a signal? Journal of Banking and Finance, 37(8):2938-2952.

Kaestner, R., Garrett, B., Chen, J., Gangopadhyaya, A., and Fleming, C. (2017). Effects of ACA medicaid expansions on health insurance coverage and labor supply. Journal of Policy Analysis and Management, 36(3):608-642.

Kleven, H. J., Landais, C., and Saez, E. (2013). Taxation and international migration of superstars: Evidence from the European football market. The American Economic Review, 103(5):1892-1924.

Klößner, S., Kaul, A., Pfeifer, G., and Schieler, M. (2017). Comparative politics and the synthetic control method revisited: A note on Abadie et al. (2015). Swiss Journal of Economics and Statistics, pages n/a-n/a.

Klößner, S. and Pfeifer, G. (2017). Outside the box: Using synthetic control methods as a forecasting technique. Applied Economics Letters, pages n/a-n/a.

Kreif, N., Grieve, R., Hangartner, D., Turner, A. J., Nikolova, S., and Sutton, M. (2016). Examination of the synthetic control method for evaluating health policies with multiple treated units. Health Economics, 25:1514-1528.

Liu, S. (2015). Spillovers from universities: Evidence from the land-grant program. Journal of Urban Economics, 87:25-41.

Montalvo, J. G. (2011). Voting after the bombings: A natural experiment on the effect of terrorist attacks on democratic elections. The Review of Economics and Statistics, 93(4):1146-1154.

Nannicini, T. and Billmeier, A. (2011). Economies in transition: How important is trade openness for growth? Oxford Bulletin of Economics and Statistics, 73(3):287-314.

O’Neill, S., Kreif, N., Grieve, R., Sutton, M., and Sekhon, J. S. (2016). Estimating causal effects: considering three alternatives to difference-in-differences estimation. Health Services and Outcomes Research Methodology, 16:1-21.

Stearns, J. (2015). The effects of paid maternity leave: Evidence from temporary disability insurance. Journal of Health Economics, 43:85-102.