Should we drop covariate cells with attrition problems?

Ferman, Bruno and Ponczek, Vladimir (2017): Should we drop covariate cells with attrition problems?

Abstract

It is well known that sample attrition can lead to inconsistent treatment effect estimators even in randomized control trials. Standard solutions to attrition problems either rely on strong assumptions on the attrition mechanisms or consider the estimation of bounds, which may be uninformative if attrition problems are severe. In this paper, we analyze strategies of focusing the analysis on subsets of the data with less observed attrition problems. We show that these strategies are asymptotically valid when the number of observations in each covariate cell goes to infinity. However, they can lead to important distortions when the number of observations per covariate cell is finite.

• Item Type: MPRA Paper
• Language: English
• Keywords: impact evaluation, attrition, partial identification
• Subjects: C - Mathematical and Quantitative Methods > C0 - General > C01 - Econometrics; C - Mathematical and Quantitative Methods > C9 - Design of Experiments > C93 - Field Experiments

References

Ahn, Hyungtaik and James L. Powell, “Semiparametric estimation of censored selection models with a nonparametric selection mechanism,” Journal of Econometrics, 1993, 58 (1), 3 – 29.

Andrews, Donald W. K. and Marcia M. A. Schafgans, “Semiparametric Estimation of the Intercept of a Sample Selection Model,” The Review of Economic Studies, 1998, 65 (3), 497–517.

Blanco, German, Carlos A. Flores, and Alfonso Flores-Lagunes, “Bounds on Average and Quantile Treatment Effects of Job Corps Training on Wages,” Journal of Human Resources, 2013, 48 (3), 659–701.

Blanco, German, Carlos A. Flores, and Alfonso Flores-Lagunes, “The Effects of Job Corps Training on Wages of Adolescents and Young Adults,” The American Economic Review, 2013, 103 (3), 418–422.

Bruhn, Miriam and David McKenzie, “In Pursuit of Balance: Randomization in Practice in Development Field Experiments,” American Economic Journal: Applied Economics, October 2009, 1 (4), 200–232.

Christensen, Garret and Edward Miguel, “Transparency, Reproducibility, and the Credibility of Economics Research,” Technical Report dec 2016.

Coffman, Lucas C. and Muriel Niederle, “Pre-analysis Plans Have Limited Upside, Especially Where Replications Are Feasible,” Journal of Economic Perspectives, 2015, 29 (3), 81–98.

Das, Mitali, Whitney K. Newey, and Francis Vella, “Nonparametric Estimation of Sample Selection Models,” The Review of Economic Studies, 2003, 70 (1), 33–58.

Flores, Carlos A., Alfonso Flores-Lagunes, Arturo Gonzalez, and Todd C. Neumann, “Estimating the Effects of Length of Exposure to Instruction in a Training Program: The Case of Job Corps,” The Review of Economics and Statistics, 2012, 94 (1), 153–171.

FLORES-LAGUNES, ALFONSO, ARTURO GONZALEZ, and TODD NEUMANN, “LEARN- ING BUT NOT EARNING? THE IMPACT OF JOB CORPS TRAINING ON HISPANIC YOUTH,” Economic Inquiry, 2010, 48 (3), 651–667.

Frumento, Paolo, Fabrizia Mealli, Barbara Pacini, and Donald B. Rubin, “Evaluating the Effect of Training on Wages in the Presence of Noncompliance, Nonemployment, and Missing Outcome Data,” Journal of the American Statistical Association, 2012, 107 (498), 450–466.

Heckman, James, “Varieties of Selection Bias,” The American Economic Review, 1990, 80 (2), 313–318.

Heckman, James J., “Sample Selection Bias as a Specification Error,” Econometrica, 1979, 47 (1), 153–161.

Horowitz, Joel L. and Charles F. Manski, “Nonparametric Analysis of Randomized Experiments with

Missing Covariate and Outcome Data,” Journal of the American Statistical Association, 2000, 95 (449), 77–84.

Imbens, Guido W. and Charles F. Manski, “Confidence Intervals for Partially Identified Parameters,” Econometrica, 2004, 72 (6), 1845–1857.

King, Gary, Emmanuela Gakidou, Nirmala Ravishankar, Ryan T. Moore, Jason Lakin, Manett Vargas, Martha Mar´ıa T´ellez-Ro jo, Juan Eugenio Hern´andez A´ vila, Mauricio Hern´andez A´ vila, and H´ector Hern´andez Llamas, “A ”Politically Robust” Experimental Design for Public Policy Evaluation, with Application to the Mexican Universal Health Insurance Program,” Journal of Policy Analysis and Management, 2007, 26, 479–506.

Lee, David S., “Training, Wages, and Sample Selection: Estimating Sharp Bounds on Treatment Effects,” The Review of Economic Studies, 2009, 76 (3), 1071.

Olken, Benjamin A., “Promises and Perils of Pre-analysis Plans,” Journal of Economic Perspectives, 2015, 29 (3), 61–80.

Rubin, Donald B., “Inference and missing data,” Biometrika, 1976, 63 (3), 581.

Schochet, Peter Z., John Burghardt, and Sheena McConnell, “Does Job Corps Work? Impact Findings from the National Job Corps Study,” American Economic Review, December 2008, 98 (5), 1864–86.

Zhang, Junni L. and Donald B. Rubin, “Estimation of Causal Effects via Principal Stratification When Some Outcomes are Truncated by ?Death?,” Journal of Educational and Behavioral Statistics, 2003, 28 (4), 353–368.