The synthetic control method: Small T, small N Monte Carlo evidence and an application to the effects of privatizing probation services on revoke rates

Süß, Philipp (2016): The synthetic control method: Small T, small N Monte Carlo evidence and an application to the effects of privatizing probation services on revoke rates.

Abstract

Relying on synthetic controls to estimate treatment effects recently gained popularity in applied econometrics. The small sample properties of the synthetic control estimator are however not suffciently investigated and even the proofs of consistency impose a factor model and require either the pre-treatment period or the pre-treatment period and the size of the donor pool going to infinity. Since applications often ignore the lack of statistical foundation in small samples, a small T small N Monte Carlo study covering standard econometric models like the ones from Differences-in-Differences setups, heterogeneous ADLX models with and without unit roots and random coeffcient models is conducted. The results suggest that the estimator is frequently unbiased, that unit roots are problematic and that a main placebo test has good size and mediocre power properties. Furthermore, the synthetic control method is used to estimate the causal effect of outsourcing probation and parole services on revoke rates by exploiting a natural experiment in Germany. Results provide evidence against increases in revoke rates due to outsourcing.

• Item Type: MPRA Paper
• Language: English
• Keywords: Synthetic control method, Finite sample properties, Unit roots, Probation, Parole, Privatization
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C13 - Estimation: General; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C23 - Panel Data Models ; Spatio-temporal Models; H - Public Economics > H1 - Structure and Scope of Government > H11 - Structure, Scope, and Performance of Government; K - Law and Economics > K4 - Legal Procedure, the Legal System, and Illegal Behavior > K40 - General

References

[1] Abadie, A. 2015. Using Synthetic Controls to Evaluate an International Strategic Positioning Program in Uruguay: Feasibility, Data Requirements, and Methodological Aspects, Study commissioned by the Inter-American Development Bank. http://idbdocs.iadb.org/wsdocs/getdocument.aspx?docnum=37312864, last accessed 11/01/2016.

[2] Abadie, A., Diamond, A., and Hainmueller, J. 2010. Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California's Tobacco Control Program. Journal of the American Statistical Association 105(490): 493-505.

[3] Abadie, A., Diamond, A., and Hainmueller, J. 2015. Comparative Politics and the Synthetic Control Method. American Journal of Political Science 59(2): 495-510.

[4] Abadie, A., and Gardeazabal, J. 2003. The Economic Costs of Conflict: A Case Study of the Basque Country. American Economic Review 93(1): 112-132.

[5] Bohn, S., Lofstrom, M., and Raphael, S. 2014. Did the 2007 Legal Arizona Workers Act reduce the state's unauthorized immigrant population? Review of Economics and Statistics 96(2): 258-269.

[6] Carvalho, C. V., Masini, R., and Medeiros, M. C. 2014. Synthetic Control Theory and Inference for Stationary Processes. NBER-NSF Time Series Conference, Federal reserve Bank of St. Louis 2014.

[7] Deering, J., Feilzer M., and Holmes, T. 2014. The transition from public to private in probation: Values and attitudes of managers in the private sector. Probation Journal 61(3):234-250.

[8] Dölling, D., Hermann, D., and Entorf, H. 2014. Evaluation der Bewährungs- und Gerichtshilfe sowie des Täter-Opfer-Ausgleichs in Baden Württemberg - Abschlussbericht. http://www.uniheidelberg.de/institute/fak2/krimi/Evaluation%20der%20BWH_GH_TOA.pdf, last accessed 11/01/2016.

[9] Gobillon, L., and Magnac, T. 2015. Regional Policy Evaluation: Interactive Fixed Effects and Synthetic Controls. Review of Economics and Statistics (forthcoming).

[10] Kaul, A., Klößner, S., Pfeifer, G., Schieler M. 2015. Synthetic Control Methods: Never Use All Pre-Intervention Outcomes as Economic Predictors. http://www.oekonometrie.unisaarland.de/papers/SCM_Predictors.pdf Version: May 27 2015, last accessed 11/01/2016.

[11] Ministry of Justice Baden Württemberg. 2014. Evaluation der Bewährungs- und Gerichtshilfe sowie des Täter-Opfer-Ausgleichs in Baden Württemberg. Justizministerium Baden Württemberg 2014.

[12] Ministry of Justice United Kingdom. 2013. Transforming Rehabilitation: A revolution in the way we manage offenders. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/228580/8517.pdf, last accessed 11/01/2016.

[13] Im, K. S., Pesaran, M. H., and Shin, Y. 2003. Testing for unit roots in heterogeneous panels. Journal of Econometrics 115(1): 53-74.

[14] Pesaran, M. H. 2007. A simple panel unit root test in the presence of cross-section dependence. Journal of Applied Econometrics 22(2): 265-312.

[15] Robinson, G., Burke, L., and Millings, M. 2016. Criminal justice identitites in transition: The case of devolved probation services in England and Wales. British Journal of Criminology 56(1): 161-178.

[16] Schloss, C. S., and Alarid, L. F. 2007. Standards in the Privatization of Probation Services: A statutory Analysis. Criminal Justice Review 32(3): 233-245.

[17] Teague, M. 2011. Probation in America: Armed, private and unaffordable? Probation Journal 58(4): 317-332.

[18] Xu, Y. 2015. Generalized Synthetic Control Method for Causal Inference with Time-Series Cross-Sectional Data. MIT Political Science Department Research Paper No. 2015-1