Methods for Evaluating Innovative Health Programs (EIHP): A Multi-Country Study

Thomas, Ranjeeta and Jones, Andrew M and Squire, Lyn (2010): Methods for Evaluating Innovative Health Programs (EIHP): A Multi-Country Study. Published in: Journal of Development Effectiveness , Vol. 2, No. 4 (December 2010): pp. 504-520.

Abstract

Designed as a global research initiative, the EIHP project aims at adding to the evidence base of health interventions that have the potential to improve health outcomes in Africa and Asia. The project focuses on rigorous, quantitative evaluations of innovative local initiatives that address the Millennium Development Goals for health: reductions in child and maternal mortality and communicable diseases. This overview brings together the outcomes and lessons from the project for evaluation methods. It draws together the methodological implications of carrying out impact evaluations under very different settings and emphasizes the need to build in evaluations in project designs.

• Item Type: MPRA Paper
• Language: English
• Keywords: Millennium Development Goals; child and maternal health; communicable diseases; impact evaluation; capacity building; Asia; Africa; Latin America
• Subjects: C - Mathematical and Quantitative Methods > C3 - Multiple or Simultaneous Equation Models ; Multiple Variables > C31 - Cross-Sectional Models ; Spatial Models ; Treatment Effect Models ; Quantile Regressions ; Social Interaction Models; H - Public Economics > H5 - National Government Expenditures and Related Policies > H53 - Government Expenditures and Welfare Programs

References

Admassie, A., Abebaw, D. and Woldemichael, A.D., 2009. Impact evaluation of the Ethiopian Health Services Extension Program. Journal of Development Effectiveness, 1(4), 430-449.

Aggarwal, A., 2010. Impact evaluation of India’s ‘Yeshasvini’ community based health insurance program. Health Economics, in press, 10.1002/hec.1605.

Arcand, J-L. and Wouabe, E.D., 2010. Teacher Training and HIV/AIDS prevention in West Africa: Regression discontinuity design evidence from Cameroon. Health Economics, in press.

Ashenfelter, O., 1978. Estimating the effect of training programs on earnings. Review of Economics and Statistics, 60, 47-57.

Ashenfelter, O. and Card, D., 1985. Using the longitudinal structure of earnings to estimate the effect of training programs. Review of Economics and Statistics, 67, 648-60.

Baird, S., Chirwa, E., McIntosh, C. and Ozler, B., 2010. The short-term impacts of a schooling conditional cash transfer program on the sexual behaviour of young women. Health Economics, in press, DOI: 10.1002/hec.1569.

Banerjee, A. V. and Duflo E., 2008 The experimental approach to development economics. NBER working paper w14467. National Bureau of Economic Research, Inc.

Basinga, P., Gertler, P.J., Binagwaho, A., Soucat, A.L.B., Sturdy J.R. and Vermeersch, C.M.J., 2009. Impact of performance based financing in Rwanda: health facility level analysis. GDN working paper 32. Global Development Network.

Blundell, R. and Costa-Dias, M., 2008. Alternative approaches to evaluation in empirical microeconomics. CEMMAP working paper CWP26/08. Centre for Microdata Methods and Practice, Institute for Fiscal Studies.

Bourguignon, F. and Sundberg, M., 2007. Aid effectiveness - opening the black box. American Economic Review, 97, 316–321.

Chandoevwit, W. and Vacharanukulkieti, K., 2009. An evaluation of a safe motherhood hospital program. GDN working paper 23. Global Development Network.

Chen, S. and Ravallion, M., 2007. Absolute poverty measures for the developing world, 1981-2004. Policy Research Working Paper WPS4211. The World Bank.

Cochran, W. and Rubin, D., 1973. Controlling bias in observational studies: a review. Sankhya: The Indian Journal of Statistics, Series A, 35(4), 417-46.

Correa C.J., Pinto, D., Camacho, J., Quintero, J., Rondón, M. and Salas, L., 2010. A couple of squirts a day keep the doctor away: a cluster randomized controlled trial of alcohol based hand sanitizer gel for prevention of infectious diseases in children. GDN working paper 39. Global Development Network.

Deaton, A.S., 2008. Instruments of development: randomization in the tropics, and the search for the elusive keys to economic development. NBER working paper w14690. National Bureau of Economic Research, Inc.

Deheija, R. and Wahba, S., 1999. Causal effects in nonexperimental studies: re-evaluating the evaluation of training programs. Journal of the American Statistical Association, 94, 1053-62.

Deheija, R. and Wahba, S., 2002. Propensity score matching methods for nonexperimental causal studies. Review of Economic Studies, 84, 151-61.

Díaz, J.J. and Jaramillo, M., 2009 Evaluating interventions to reduce maternal mortality: evidence from Peru's PARSalud programme. Journal of Development Effectiveness, 1(4), 387-412.

Duflo, E., Glennerster, R., and Kremer, M., 2007.Using Randomization in Development Economics Research: A Toolkit. In: T.P. Schultz and J.A. Strauss, ed. Handbook of Development Economics. Amsterdam: Elsevier, vol. 4, chap. 61, 3895–3962.

Feng, X.L., Shi, G., Wang, Y., Xu, L., Luo, H., Shen, J., Yin, H. and Guo, Y., 2010. An impact evaluation of the Safe Motherhood Program in China. Health Economics, in press, DOI: 10.1002/hec.1593.

Heckman, J. J., 2008. Econometric causality. CEMMAP working paper CWP1/08. Centre for Microdata Methods and Practice, Institute for Fiscal Studies.

Heckman, J.J., Ichimura, H. and Todd, P. E., 1998. Matching as an econometric evaluation estimator. Review of Economic Studies, 65, 261-94.

Heckman, J.J. and Robb, R., 1985. Alternative models for evaluating the impact of interventions, In: J. J. Heckman and B. Singer (eds.), Longitudinal Analysis of Labour Market Data. Cambridge: Cambridge University Press, chap. 4, 156

Heckman, J.J., and Vytlacil, E., 2007. Econometric evaluation of social programs. In: J,J Heckman and E. Leamer, ed. Handbook of Econometrics, Amsterdam: Elsevier, vol 6B.

Imbens, G.W. and Wooldridge, J.M., 2008. Recent developments in the econometrics of program evaluation. NBER Working Paper 14251. National Bureau of Economic Research, Inc.

Holland, P.W., 1986. Statistics and causal inference. Journal of the American Statistical Association, 81, 945-60.

Lalonde, R.J., 1986. Evaluating the econometric evaluations of training programs with experimental data. American Economic Review, 76, 604-20.

Mensah, J., Oppong, J.R. and Schmidt, C.M., 2010 Ghana’s National Health Insurance Scheme in the Context of the Health MDGS – An Empirical Evaluation Using Propensity Score Matching. Health Economics, in press.

Ngo, A., Phan, H., Pham, V., Trinh, T. and Truong, K., 2009. Impacts of a government social franchise model on perceptions of service quality and client satisfaction at commune health stations in Vietnam. Journal of Development Effectiveness, 1(4),413-429.

Ngoc, U.V., and Huy, V.Q., 2010. Health impact evaluation: Project Young Medical Volunteers for Vietnam Rural Mountain. GDN working paper 38. Global Development Network.

Nizalova, O. and Vyshnya, M., 2010. Evaluation of the impact of the Mother and Infant Health Project in Ukraine. Health Economics, in press, DOI: 10.1002/hec.1609.

Oduor, J., Kamau, A. and Mathenge, E., 2009. Evaluating the impact of micro-franchising the distribution of anti-malarial drugs in Kenya on malaria mortality and morbidity. Journal of Development Effectiveness, 1(3), 353-377.

Ravillion, M., 2007. Evaluating Anti-Poverty Programs. In: T. P. Schultz and J. A. Strauss, ed. Handbook of Development Economics, Amsterdam: Elsevier, vol. 4, chap. 59, 3787-3846.

Rocha, R. and Soares, R., 2010. Evaluating the impact of community based health interventions: Evidence from Brazil’s Family Health Program. Health Economics, in press, DOI: 10.1002/hec.1607.

Rosenbaum, P.R. and Rubin, D.B., 1983. The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41-55.

Rosenbaum, P.R. and Rubin, D.B., 1984. Reducing the bias in observational studies using subclassification on the propensity score. Journal of the American Statistical Association, 79, 516-24.

Rubin, D. B., 1973a. Matching to remove bias in observational studies. Biometrics, 29, 159-83.

Rubin, D.B., 1973b. The use of matched sampling and regression adjustments to remove bias in observational studies. Biometrics, 29, 185-203.

Rubin, D.B., 1974. Estimating causal effects of treatments in randomized and non-randomized studies. Journal of Educational Psychology, 66, 688-701.

Rubin, D.B., 2006. Matched sampling for causal effects. Cambridge: Cambridge University Press.

Salehi-Isfahani, D., Abbasi-Shavazi, M.J. and Hosseini-Chavoshi, M., 2010. Family planning and rural fertility decline in Iran: A study in program evaluation. Health Economics, in press, DOI: 10.1002/hec.1613.

Simwaka, B. N., Simwaka, K. and Bello, G., 2009. Retrospective analysis of a school-based malaria treatment programme demonstrates a positive impact on health and education outcomes in Mangochi district, Malawi. Journal of Development Effectiveness, 1(4), 492–506.

Teerawattananon, Y., Leelukhanaveera, Y., Thavorncharoensap, M., Hanvoravovongchai, P., Ingsrisawang, L., Tantivess, S., Chaikledkaew, U., Mohara, A., Lertpiriyasuwat, C. and Pimsawan, N., 2009. Provider-initiated HIV/AIDS counselling and testing at healthcare facilities in Thailand: a cluster-randomization trial. Journal of Development Effectiveness, 1(4), 450-469.

Thornton, R., Hatt, L., Islam, M., Field, E., Solís, F. and González Moncada, M.A., 2010. Social security health insurance for the informal sector in Nicaragua: A randomized evaluation. Health Economics, in press, DOI: 10.1002/hec.1635.

United Nations Department of Economic and Social Affairs, 2008. Millennium Development Goals Report.

White, H., 2006. Impact evaluation: The experience of the Independent Evaluation Group of the World Bank. Technical Report 1111. The World Bank.