Revisiting the Synthetic Control Estimator

Ferman, Bruno and Pinto, Cristine (2016): Revisiting the Synthetic Control Estimator.

Abstract

The synthetic control (SC) method has been recently proposed as an alternative to estimate treatment effects in comparative case studies. The SC relies on the assumption that there is a weighted average of the control units that reconstructs the potential outcome of the treated unit in the absence of treatment. If these weights were known, then constructing the counterfactual for the treated unit using a weighted average of the control units would provide an unbiased estimator for the treatment effect, even if selection into treatment is correlated with the unobserved heterogeneity. In this paper, we revisit the SC method in a linear factor model where the SC weights are considered nuisance parameters that are estimated to construct the SC estimator. We show that, when the number of control units is fixed, the estimated SC weights will generally not converge to the weights that reconstruct the factor loadings of the treated unit, even when the number of pre-intervention periods goes to infinity. As a consequence, the SC estimator will be asymptotically biased if treatment assignment is correlated with the unobserved heterogeneity. The asymptotic bias only vanishes when the variance of the idiosyncratic error goes to zero. We suggest a slight modification in the SC method that guarantees that the SC estimator is asymptotically unbiased and has a lower asymptotic variance than the difference-in-differences (DID) estimator when the DID identification assumption is satisfied. We also propose an alternative way to estimate the SC weights that provides an asymptotically unbiased estimator under additional assumptions on the error structure. Finally, we consider the implications of our findings to the permutation test suggested in Abadie et al. (2010).

• Item Type: MPRA Paper
• Language: English
• Keywords: synthetic control, difference-in-differences; linear factor model, inference, permutation test
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C12 - Hypothesis Testing: General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C13 - Estimation: General; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C21 - Cross-Sectional Models ; Spatial Models ; Treatment Effect Models ; Quantile Regressions; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C23 - Panel Data Models ; Spatio-temporal Models

References

Abadie, Alberto, Alexis Diamond, and Jens Hainmueller, “Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program,” Journal of the American Statiscal Association, 2010, 105 (490), 493–505.

Abadie, Alberto, Alexis Diamond, and Jens Hainmueller, “Comparative Politics and the Synthetic Control Method,” American Journal of Political Science, 2015, 59 (2), 495–510.

Abadie, Alberto, and Javier Gardeazabal, “The Economic Costs of Conflict: A Case Study of the Basque Country,” American Economic Review, 2003, 93 (1), 113–132.

Acemoglu, Daron, Simon Johnson, Amir Kermani, James Kwak, and Todd Mitton, “The Value of Connections in Turbulent Times: Evidence from the United States,” December 2013. NBER Working Paper 19701.

Ando, Michihito, “Dreams of Urbanization: Quantitative Case Studies on the Local Impacts of Nuclear Power Facilities using the Synthetic Control Method,” Journal of Urban Economics, June 2015, 85, 68–85.

Ando, Michihito, and Fredrik S¨avje, “Hypothesis Testing with the Synthetic Control Method,” 2013. Working Paper.

Athey, Susan and Guido Imbens, “The State of Applied Econometrics-Causality and Policy Evaluation,” arXiv preprint arXiv:1607.00699, 2016.

Barone, Guglielmo and Sauro Mocetti, “Natural Disasters, Growth and Institutions: a Tale of Two Earthquakes,” Journal of Urban Economics, 2014, pp. 52–66.

Bauhoff, Sebastian, “The Effect of School Nutrition Policies on Dietary Intake and Overweight: a Synthetic Control Approach,” Economics and Human Biology, 2014, pp. 45–55.

Belot, Michele and Vincent Vandenberghe, “Evaluating the Threat Effects of Grade Repetition: Ex- ploiting the 2001 Reform by the French-Speaking Community of Belgium,” Education Economics, 2014, 22 (1), 73–89.

Billmeier, Andreas and Tommaso Nannicini, “Trade Openness and Growth: Pursuing Empirical Glasnost,” IMF Staff Papers, 2009, 56 (3), 447–475.

Billmeier, Andreas and Tommaso Nannicini, “Assessing Economic Liberalization Episodes: A Synthetic Control Approach,” The Review of Economics and Statistics, 2013, 95 (3), 983–1001.

Bohn, Sarah, Magnus Lofstrom, and Steven Raphael, “Did the 2007 Legal Arizona Workers Act Reduce the State’s Unauthorized Immigrant Population?,” The Review of Economics and Statistics, 2014, 96 (2), 258–269.

Bove, Vincenzo, Leandro Elia, and Ron P. Smith, “The Relationship between Panel and Synthetic Control Estimators on the Effect of Civil War,” October 2014. Working Paper.

Calderon, Gabriela, “The Effects of Child Care Provision in Mexico,” July 2014. Working paper.

Canay, Ivan A., Joseph P. Romano, and Azeem M. Shaikh, “Randomization Tests under an Approximate Symmetry Assumption?,” 2014.

Carrasco, Vinicius, Joao M. P. de Mello, and Isabel Duarte, “A D´ecada Perdida: 2003 – 2012,” 2014. Texto para Discussa˜o,.

Carvalho, Carlos V., Ricardo Mansini, and Marcelo C. Medeiros, “ArCo: An Artificial Counter- factual Approach for Aggregate Data,” February 2015. Working Paper.

Cavallo, Eduardo, Sebastian Galiani, Ilan Noy, and Juan Pantano, “Catastrophic Natural Disasters and Economic Growth,” The Review of Economics and Statistics, 2013, 95 (5), 1549–1561.

Chan, Ho Fai, Bruno S. Frey, Jana Gallus, and Benno Torgler, “Academic Honors and Performance,” Labour Economics, 2014, 31, 188–204.

Chernozhukov, Victor, Han Hong, and Elie Tamer, “Estimation and Confidence Regions for Parameter Sets in Econometric Models,” Econometrica, 2007, 75 (5), 1243–1284.

Coffman, Makena and Ilan Noy, “Hurricane Iniki: Measuring the Long-Term Economic Impact of Natural Disaster Using Synthetic Control,” Environment and Development Economics, 2011, 17, 187–205.

de Souza, Fernando Friac¸a Asmar, “Tax Evasion and Inflation: Evidence from the Nota Fiscal Paulista Program,” Master’s thesis, Pontif´ıcia Universidade Cat´olica March 2014.

Dhungana, Sandesh, “Identifying and Evaluating Large Scale Policy Interventions: What Questions Can We Answer?,” December 2011.

Doudchenko, Nikolay and Guido Imbens, “Balancing, regression, difference-in-differences and synthetic control methods: A synthesis,” 2016.

DuPont, William and Ilan Noy, “What Happened to Kobe? A Reassessment of the Impact of the 1995 Earthquake in Japan,” March 2012.

Ferman, Bruno and Cristine Pinto, “Inference in Differences-in-Differences with Different Group Sizes,” 2016. Working Paper.

Ferman, Bruno and Cristine Pinto, and Vitor Possebom, “Cherry Picking with Synthetic Controls,” 2016. Working Paper.

Firpo, Sergio and Vitor Possebom, “Synthetic Control Estimator: A Generalized Inference Procedure and Confidence Sets,” April 2016.

Gathani, Sachin, Massimiliano Santini, and Dimitri Stoelinga, “Innovative Techniques to Evaluate the Impacts of Private Sector Developments Reforms: An Application to Rwanda and 11 other Countries.” Working Paper.

Gobillon, Laurent and Thierry Magnac, “Regional Policy Evaluation: Interactive Fixed Effects and Synthetic Controls,” PSE Working Papers halshs-00849071, HAL July 2013.

Gobillon, Laurent and Thierry Magnac, “Regional Policy Evaluation: Interative Fixed Effects and Synthetic Controls,” Review of Eco- nomics and Statistics, 2016. Forthcoming.

Hinrichs, Peter, “The Effects of Affirmative Action Bans on College Enrollment, Educational Attainment, and the Demographic Composition of Universities,” Review of Economics and Statistics, March 2012, 94 (3), 712–722.

Hosny, Amr Sadek, “Algeria’s Trade with GAFTA Countries: A Synthetic Control Approach,” Transition Studies Review, 2012, 19, 35–42.

Jinjarak, Yothin, Ilan Noy, and Huanhuan Zheng, “Capital Controls in Brazil — Stemming a Tide with a Signal?,” Journal of Banking & Finance, 2013, 37, 2938–2952.

Kaul, Ashok, Stefan Kl¨obner, Gregor Pfeifer, and Manuel Schieler, “Synthetic Control Methods: Never Use All Pre-Intervention Outcomes as Economic Predictors,” May 2015. Working Paper.

Kirkpatrick, A. Justin and Lori S. Bennear, “Promoting Clean Enery Investment: an Empirical Analysis of Property Assessed Clean Energy,” Journal of Environmental Economics and Management, 2014, 68, 357–375.

Kleven, Henrik Jacobsen, Camille Landais, and Emmanuel Saez, “Taxation and International Migration of Superstars: Evidence from European Football Market,” American Economic Review, 2013, 103 (5), 1892–1924.

Kreif, No´emi, Richard Grieve, Dominik Hangartner, Alex James Turner, Silviya Nikolova, and Matt Sutton, “Examination of the Synthetic Control Method for Evaluating Health Policies with Multiple Treated Units,” Health Economics, 2015.

Li, Qi, “Economics Consequences of Civil Wars in the Post-World War II Period,” The Macrotheme Review, 2012, 1 (1), 50–60.

Liu, Shimeng, “Spillovers from Universities: Evidence from the Land-Grant Program,” Journal of Urban Economics, 2015, 87, 25–41.

Mideksa, Torben K., “The Economic Impact of Natural Resources,” Journal of Environmental Economics and Management, 2013, 65, 277–289.

Montalvo, Jos´e G., “Voting after the Bombings: A Natural Experiment on the Effect of Terrorist Attacks on Democratic Elections,” Review of Economics and Statistics, 2011, 93 (4), 1146–1154.

Newey, Whitney K. and Daniel McFadden, “Chapter 36 Large sample estimation and hypothesis testing,” in “in,” Vol. 4 of Handbook of Econometrics, Elsevier, 1994, pp. 2111 – 2245.

Pinotti, Paolo, “Organized Crime, Violence and the Quality of Politicians: Evidence from Southern Italy,” 2012.

Pinotti, Paolo, “The Economic Costs of Organized Crime: Evidence from Southern Italy,” April 2012. Temi di Discus- sione (Working Papers).

Powell, David, “Synthetic Control Estimation Beyond Case Studies: Does the Minimum Wage Reduce Employment?,” 2016. RAND Corporation, WR-1142.

Ribeiro, Felipe, Guilherme Stein, and Thomas Kang, “The Cuban Experiment: Measuring the Role of the 1959 Revolution on Economic Performance using Synthetic Control,” May 2013.

Sanso-Navarro, Marcos, “The effects on American Foreign Direct Investment in the United Kingdom from Not Adopting the Euro,” Journal of Common Markets Studies, 2011, 49 (2), 463–483.

Saunders, Jessica, Russel Lundberg, Anthony A. Braga, Greg Ridgeway, and Jeremy Miles, “A Synthetic Control Approach to Evaluating Place-Based Crime Interventions,” Journal of Quantitative Criminology, 2014.

Severnini, Edson R., “The Power of Hydroelectric Dams: Agglomeration Spillovers,” March 2014. IZA Discussion Paper, No. 8082.

Sills, Erin O., Diego Herrera, A. Justin Kirkpatrick, Amintas Brandao, Rebecca Dickson, Simon Hall, Subhrendu Pattanayak, David Shoch, Mariana Vedoveto, Luisa Young, and Alexander Pfaff, “Estimating the Impact of a Local Policy Innovation: The Synthetic Control Method Applied to Tropica Desforestation,” PLOS One, 2015.

Smith, Brock, “The Resource Curse Exorcised: Evidence from a Panel of Countries,” Journal of Develop- ment Economics, 2015, 116, 57–73.

Wong, Laurence, “Three Essays in Causal Inference.” PhD dissertation, Stanford University March 2015.

Xu, Yiqing, “Regional Policy Evaluation: Interative Fixed Effects and Synthetic Controls,” Political Anal- ysis, 2016. Forthcoming.

Yu, Jingwen and Chunchao Wang, “Political Risk and Economic Development: A Case Study of China,” Eknomska Istrazianja - Economic Research, 2013, 26 (2), 35–50.