An Introduction to Alternative Methods in Program Impact Evaluation

Nguyen Viet, Cuong (2006): An Introduction to Alternative Methods in Program Impact Evaluation. Published in: Mansholt Working Papers , Vol. Year 2, No. No. 33 (1 June 2007): pp. 1-50.

Abstract

This paper presents an overview of several widely-used methods in program impact evaluation. In addition to a randomization-based method, these methods are categorized into: (i) methods assuming “selection on observable” and (ii) methods assuming “selection on unobservable”. The paper discusses each method under identification assumptions and estimation strategy. Identification assumptions are presented in a unified framework of counterfactual and two equation model. Finally, the paper uses simulated data to illustrate how these methods work under different identification assumptions.

• Item Type: MPRA Paper
• Language: English
• Keywords: Program impact evaluation, treatment effect, counterfactual, potential outcomes, selection on observable, selection on unobservable.
• Subjects: C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C20 - General; I - Health, Education, and Welfare > I3 - Welfare, Well-Being, and Poverty > I38 - Government Policy ; Provision and Effects of Welfare Programs; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C21 - Cross-Sectional Models ; Spatial Models ; Treatment Effect Models ; Quantile Regressions

References

Abadie, A. and G. W. Imbens (2002), “Simple and Bias Corrected Matching Estimators for Average Treatment Effects,” unpublished manuscript.

Buddelmeyer, H. and E. Skoufias (2004), “An Evaluation of the Performance of Regression Discontinuity Design in Progresa.” World Bank Policy Research Working Paper No. 3386.

Cochran, W. G. (1968), “The effectiveness of Adjustment by Subclassification in Removing Bias in Observational Studies”. Biometrics 24, 295-313.

Cochran, W. G. and S. Paul Chambers (1965), “The Planning of Observational Studies of Human Population (with discussion” Journal of the Royal Statistical Society. Series A (General), Vol. 128, No. 2. (1965), pp. 234-266.

Dawid, A. P. (1979), “Conditional Independence in Statistical Theory”, J. R. Statist. Soc., 41, No. 1: 1-31.

Dehejia R. H. and S. Wahba (1998), “Propensity Score Matching Methods for Non-Experimental Causal Studies”, NBER Working Paper 6829, Cambridge, Mass.

Fan, J. (1992), “Local Linear Regression Smoothers and their Minimax Efficiencies”. The Annals of Statistics 21, 196-216.

Greene W. H. (2003), Econometric Analysis, Prentice Hall Press, Firth Edition, 2003.

Hahn, J., (1998), “On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects," Econometrica 66 (2), 315-331.

Hahn, J., P. Todd and W. van der Klaauw (2001), “Identification and Estimation of Treatment Effects with a Regression-Discontinuity Design.” Econometrica 69(1): 201-09.

Heckman, J. (1978), “Dummy Endogenous Variables in a Simultaneous Equation System" Econometrica, 46, 931-959.

Heckman, J., H. Ichimura, J. A. Smith, and P. E. Todd(1998a), “Characterizing Selection Bias Using Experimental Dada,” Econometrica, Vol. 66, 1017-1098.

Heckman, J., J. Smith, and N. Clements (1997a), “Making the Most Out of Program Evaluations and Social Experiments: Accounting for Heterogeneity in Program Impacts”, Review of Economic Studies, (1997) 64, 487 – 535.

Heckman, J., L. Lochner and C. Taber (1998a), “General Equilibrium Treatment Effects: A Study of Tuition Policy”, American Economic Review. 88(2):381-386.

Heckman, J., R. Lalonde and J. Smith (1999), "The Economics and Econometrics of Active Labor Market Programs," Handbook of Labor Economics, Volume 3, Ashenfelter, A. and D. Card, eds., Amsterdam: Elsevier Science.

Heckman, J., N. Hohmann, M. Khoo and J. Smith (2000), “Substitution and Dropout Bias In Social Experiments: Evidence from an Influential Social Experiment”, The Quarterly Journal of Economics, May 2000

Heckman, J. and E. J. Vytlacil (1999), “Local Instrumental Variables and Latent Variable Models for Identifying and Bounding Treatment Effects”, Proceeding of National Academy of Science, Vol.96, 4730-4734.

Heckman, J., H. Ichimura, and P. Todd (1997b), “Matching as an Econometric Evaluation Estimators: Evidence from Evaluating a Job Training Programme”, Review of Economic Studies, 64 (4), 605- 654.

Hirano K., G. W. Imbens and G. Ridder (2002), “Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score”, Working Paper

Imbens G. W. and J. D. Angrist (1994), “Identification and Estimation of Local Average Treatment Effect”, Econometrica, Vol. 62, No. 2, 467-475.

Mahalanobis P. C. (1936), “On the Generalized Distance in Statistics” Proc. Nat. Inst. Sci. Ind., Vol. 12 (1936), 49-55.

Moffitt, R. (1991), “Program Evaluation with Nonexperimental Data.” Evaluation Review. 15(3). 291-314.

Powell, J. (1994), “Estimation of Semiparametric Models”, in: R. Engle and D. McFadden, eds., Handbook of Econometrics, vol. 4 (North-Holland, Amsterdam, Netherlands) 2443-2521.

Rosenbaum, P. and R. Rubin (1983), “The Central Role of the Propensity Score in Observational Studies for Causal Effects”, Biometrika, 70 (1), 41-55.

Rosenbaum, P. and R. Rubin (1983), “The Central Role of the Propensity Score in Observational Studies for Causal Effects”, Biometrika, 70 (1), 41-55.

Rosenbaum, P. and R. Rubin (1984), “Reducing Bias in Observation Studies Using Subclassification on the Propensity Score”, Journal of Statistical Association, Vol. 79, No. 387, September 1984, 516-523.

Rosenbaum, P. and R. Rubin (1985a), “Constructing a Control Group Using Multivariate Matched Sampling Methods that Incorporate the Propensity Score”, American Statistician, 39 (1), 33-38.

Rosenbaum, P. and R. Rubin (1985b), “The Bias due to Incomplete Matching”, Biometrics, Vol. 41, No. 1, March 1984, 103-116.

Roy, A. (1951), “Some Thoughts on the Distribution of Earnings”, Oxford Economic Papers 3:135-146.

Rubin, D. (1974), “Estimating Causal Effects of Treatments in Randomized and Non-Randomized Studies”, Journal of Educational Psychology 66:688-701.

Rubin, D. (1977), “Assignment to a Treatment Group on the Basis of a Covariate”, Journal of Educational Statistics, 2 (1), 1-26.

Rubin, D. (1978), “Bayesian Inference for Causal Effects: The Role of Randomization”, The Annals of Statistics 6, 34-58.

Rubin, D. (1979), “Using Multivariate Sampling and Regression Adjustment to Control Bias in Observational Studies.” Journal of the American Statistical Association 74: 318–328.

Rubin, D. (1980), “Bias Reduction Using Mahalanobis-Metric Matching.” Biometrics 36 (2): 293–298.

Smith, J. and P. Todd. (2005), “Does Matching Overcome LaLonde’s Critique of Nonexperimental Estimators?” Journal of Econometrics 125 (1–2): 305–353.

Van der Klaauw, W. (2002), “Estimating the Effect of Financial Aid Offers on College Enrollment: A Regression-Discontinuity Approach.” International Economic Review 43(4): 1249-87.

Wooldridge J. M. (2001), Econometric Analysis of Cross Section and Panel Data, The MIT Press, Cambridge, Massachusetts London, England