Adaptive Experimental Design Using the Propensity Score

Hahn, Jinyong and Hirano, Keisuke and Karlan, Dean (2008): Adaptive Experimental Design Using the Propensity Score.

Abstract

Many social experiments are run in multiple waves, or are replications of earlier social experiments. In principle, the sampling design can be modified in later stages or replications to allow for more efficient estimation of causal effects. We consider the design of a two-stage experiment for estimating an average treatment effect, when covariate information is available for experimental subjects. We use data from the first stage to choose a conditional treatment assignment rule for units in the second stage of the experiment. This amounts to choosing the propensity score, the conditional probability of treatment given covariates. We propose to select the propensity score to minimize the asymptotic variance bound for estimating the average treatment effect. Our procedure can be implemented simply using standard statistical software and has attractive large-sample properties.

• Item Type: MPRA Paper
• Language: English
• Subjects: C - Mathematical and Quantitative Methods > C9 - Design of Experiments > C90 - General; ; C - Mathematical and Quantitative Methods > C9 - Design of Experiments > C93 - Field Experiments; C - Mathematical and Quantitative Methods > C0 - General > C01 - Econometrics

References

Angrist, J. D., Imbens, G. W., and Rubin, D. B., 1996, "Identification of Causal Effects Using Instrumental Variables," Journal of the American Statistical Association 91(434).

Chaloner, K., and Verdinelli, I., 1995, "Bayesian Experimental Design: A Review," Statistical Science 10(3), 273-304.

Chamberlain, G. 1986, "Asymptotic Efficiency in Semiparametric Models with Censoring," Journal of Econometrics 32, 189-218.

Flores-Lagunes, A., Gonzalez, A., and Neumann, T., 2006, "Learning But Not Earning? The Impact of Job Corps Training for Hispanics," working paper, University of Arizona.

Gertler, P., Martinez, S., and Rubio-Codina, M., 2006, "Investing Cash Transfers to Raise Long-Term Living Standards," World Bank Policy Research Paper 3994.

Green, D., and Gerber, A., 2004, "Get Out the Vote! How to Increase Voter Turnout," Washington, DC: Brookings Institution Press.

Hahn, J., 1998, "On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects," Econometrica 66(2), 315-331.

Hirano, K., Imbens, G. W., and Ridder, G., 2003, "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica 71(4), 1161-1189.

Imbens, G. W., and Angrist, J. D., 1994, "Identification and Estimation of Local Average Treatment Effects," Econometrica 62(2): 467-475.

Johnson and Simester, 2006, "Dynamic Catalog Mailing Policies," Management Science 52(5): 683-696.

Karlan, D., and List, J., 2007, "Does Price Matter in Charitable Giving? Evidence from a Large-Scale Natural Field Experiment," forthcoming, American Economic Review.

Karlan, D. S., and Zinman, J., 2006, "Credit Elasticities in Less-Developed Economies: Implications for Microfinance," working paper, Yale University.

Manski, C. F., and McFadden, D. L., 1981, "Alternative Estimators and Sampling Designs for Discrete Choice Analysis," Ch. 1 in Structural Analysis of Discrete Data and Econometric Applications, ed. C. F. Manski and D. L. McFadden, Cambridge: MIT Press.

Neyman, J., 1934, "On the Two Aspects of the Representative Method: The Method of Stratified Sampling and the Method of Purposive Selection," Journal of the Royal Statistical Society Series A, 97, 558-625.

Rosenbaum, P. R., and Rubin, D. B., 1983, "The Central Role of the Propensity Score in Observational Studies for Causal Effects," Biometrika 70(1): 41-55.

Solomon, H., and Zacks, S., 1970, "Optimal Design of Sampling from Finite Populations: A Critical Review and Indication of New Research Areas," Journal of the American Statistical Association 65(330), 653-677.

Sukhatme, P. V., 1935, "Contributions to the Theory of the Representative Method," Journal of the Royal Statistical Society, Supplement 2, 253-268.