Pitfalls in the Development of Falsification Tests: An Illustration from the Recent Minimum Wage Literature

Clemens, Jeffrey (2017): Pitfalls in the Development of Falsification Tests: An Illustration from the Recent Minimum Wage Literature.

Abstract

This paper examines a ``falsification test'' from the recent minimum wage literature. The analysis illustrates several pitfalls associated with developing and interpreting such exercises, which are increasingly common in applied empirical work. Clemens and Wither (2014) present evidence that minimum wage increases contributed to the magnitude of employment declines among low-skilled groups during the Great Recession. Zipperer (2016) presents regressions that he interprets as falsification tests for Clemens and Wither's baseline regression. He interprets his results as evidence that Clemens and Wither's estimates are biased. In this paper, I demonstrate that Zipperer's falsification tests are uninformative for their intended purpose. The properties of clustered robust standard errors do not carry over from Clemens and Wither's baseline specification (27 treatment states drawn from 50) to Zipperer's falsification tests (3 or 5 ``placebo treatment'' states drawn from 23). Confidence intervals calculated using a setting-appropriate permutation test extend well beyond the tests' point estimates. Further, I show that the sub-samples to which Zipperer's procedure assigns ``placebo treatment status'' were disproportionately affected by severe housing crises. His test's point estimates are highly sensitive to the exclusion of the most extreme housing crisis experiences from the sample. An inspection of data on the housing market, prime aged employment, overall unemployment rates, and aggregate income per capita reveals the test's premise that regional neighbors form reasonable counterfactuals to be incorrect in this setting.

• Item Type: MPRA Paper
• Language: English
• Keywords: Falsification Test; Program Evaluation; Minimum Wage
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C18 - Methodological Issues: General; J - Labor and Demographic Economics > J2 - Demand and Supply of Labor; J - Labor and Demographic Economics > J3 - Wages, Compensation, and Labor Costs

References

[1] Abadie, A., A. Diamond, and J. Hainmueller (2012): “Synthetic control methods for comparative case studies: Estimating the effect of Californias tobacco control program,” Journal of the American statistical Association.

[2] Allegretto, S., A. Dube, M. Reich, and B. Zipperer (2017): “Credible Research Designs for Minimum Wage Studies,” ILR Review, 70(3), 559–592.

[3] Bertrand, M., E. Duflo, and S. Mullainathan (2004): “How Much Should We Trust Differences-in-Differences Estimates?,” Quarterly Journal of Economics, 119(1).

[4] Cameron, A. C., J. B. Gelbach, and D. L. Miller (2008): “Bootstrap-based improvements for inference with clustered errors,” The Review of Economics and Statistics, 90(3), 414–427.

[5] Clemens, J. (2015): “The Minimum Wage and the Great Recession: Evidence from the Current Population Survey,” NBER Working Paper 21830.

[6] Clemens, J. (2016): “Redistribution through Minimum Wage Regulation: An Analysis of Program Linkages and Budgetary Spillovers,” in Tax Policy and the Economy, Volume 30, ed. by J. Brown. University of Chicago Press.

[7] Clemens, J. (2017): “The Minimum Wage and the Great Recession: A Response to Zipperer and Recapitulation of the Evidence,” Mimeo.

[8] Clemens, J., and M. Wither (2014): “The Minimum Wage and the Great Recession: Evidence of Effects on the Employment and Income Trajectories of Low-Skilled Workers,” NBER Working Paper 20724.

[9] Dube, A., T. W. Lester, and M. Reich (2010): “Minimum wage effects across state borders: Estimates using contiguous counties,” The review of economics and statistics, 92(4), 945–964.

[10] Imbens, G. W., and P. R. Rosenbaum (2005): “Robust, accurate confidence intervals with a weak instrument: quarter of birth and education,” Journal of the Royal Statistical Society: Series A (Statistics in Society), 168(1), 109–126.

[11] Meer, J., and J. West (2016): “Effects of the Minimum Wage on Employment Dynamics.,” Journal of Human Resources, 51(2).

[12] Neumark, D., J. I. Salas, and W. Wascher (2014a): “More on recent evidence on the effects of minimum wages in the United States,” IZA Journal of Labor policy, 3(1), 24.

[13] Neumark, D., J. I. Salas, and W. Wascher (2014b): “Revisiting the Minimum WageEmployment Debate: Throwing Out the Baby with the Bathwater?,” Industrial & Labor Relations Review, 67(3 suppl), 608–648.

[14] Neumark, D., and W. Wascher (2006): “Minimum wages and employment: A review of evidence from the new minimum wage research,” NBER Working Paper 12663.

[15] Neumark, D., and W. Wascher (2017): “Reply to Credible Research Designs for Minimum Wage Studies,” ILR Review, p. 0019793917698429.

[16] Zipperer, B. (2016): “Did the minimum wage or the Great Recession reduce low-wage employment? Comments on Clemens and Wither (2016),” Discussion paper.