Smile: A Simple Diagnostic for Selection on Observables

Slichter, David (2020): Smile: A Simple Diagnostic for Selection on Observables.

Abstract

This paper develops a simple diagnostic for the selection on observables assumption in the case of a binary treatment variable. I show that, under common assumptions, when selection on observables does not hold, designs based on selection on observables will estimate treatment effects approaching infinity or negative infinity among observations with propensity scores close to 0 or 1. Researchers can check for violations of selection on observables either informally by looking for a "smile" shape in a binned scatterplot, or with a simple formal test. When selection on observables fails, the researcher can detect the sign of the resulting bias.

• Item Type: MPRA Paper
• Language: English
• Keywords: unconfoundedness, diagnostic test
• Subjects: C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C21 - Cross-Sectional Models ; Spatial Models ; Treatment Effect Models ; Quantile Regressions; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C29 - Other

References

Black, D.A. and J.A. Smith (2004). How Robust is the Evidence on the Effects of College Quality? Evidence from Matching, Journal of Econometrics, 121, 99-124.

Blundell, R. and J. Horowitz (2007). A Non-Parametric Test of Exogeneity, Review of Economic Studies, 74, 1035-1058.

Caetano, C. (2015). A Test of Exogeneity Without Instrumental Variables in Models with Bunching, Econometrica. 83(4), 1581-1600.

Caughey, D. and J.S. Sekhon (2011). Elections and the Regression Discontinuity Design: Lessons from Close U.S. House Races, 1942-2008, Political Analysis, 19(4), 385-408.

Chen, T., Ji, Y., Zhou, Y., and P. Zhu (2018). Testing Conditional Mean Independence Under Symmetry, Journal of Business and Economic Statistics, 36(4), 615-627.

Donald, S.G., Hsu, Y., and R.P. Lieli (2014). Testing the Unconfoundedness Assumption via Inverse Probability Weighted Estimators of (L)ATT, Journal of Business and Economic Statistics, 32(3), 395-415.

Hansen, L.P. (1982). Large Sample Properties of Generalized Method of Moments Estimators, Econometrica, 50(4), 1029-1054.

Hausman, J.A. (1978). Specification Tests in Econometrics, Econometrica, 46(6), 1251-1271.

Heckman, J.J. (1979). Sample Selection Bias as a Specification Error, Econometrica, 47(1), 153-161.

Lee, D.S. (2008). Randomized Experiments from Non-Random Selection in U.S. House Elections, Journal of Econometrics, 142(2), 675-697.

Khalil, U. and N. Yildiz (2017). A Test of the Selection-on-Observables Assumption Using a Discontinuously Distributed Covariate, working paper.

Olsen, R.J. (1980). A Least Squares Correction for Selectivity Bias, Econometrica, 48(7), 1815-1820.

Rivers, D. and Q.H. Vuong (1988). Limited Information Estimators and Exogeneity Tests for Simultaneous Probit Models, Journal of Econometrics, 39, 347-366.

Rosenbaum, P.R. and D.B. Rubin (1983). The Central Role of the Propensity Score in Observational Studies for Causal Effects, Biometrika, 70(1), 41-55.

Sargan, J.D. (1958) The Estimation of Economic Relationships Using Instrumental Variables, Econometrica, 26(3), 393-415.

Young, A. (2020). Consistency without Inference: Instrumental Variables in Practical Application, working paper.