High Noon for Microfinance Impact Evaluations: Re-investigating the Evidence from Bangladesh

Duvendack, Maren and Palmer-Jones, Richard (2011): High Noon for Microfinance Impact Evaluations: Re-investigating the Evidence from Bangladesh.

Abstract

Recently, microfinance has come under increasing criticism raising questions of the validity of iconic studies which have justified the microfinance phenomenon. This paper applies propensity score matching (PSM), which has become widely used for the analysis of observational data, to the study by Pitt and Khandker (1998) which has been labelled the most rigorous evidence supporting claims that microfinance benefits the poorest especially when targeted on women. After carefully reconstructing the data we differentiate outcomes by gender of borrower, take account of borrowing from several formal and informal sources, and find that the mainly positive impacts of microfinance that we observe are shown by sensitivity analysis to be highly vulnerable to selection on unobservables, and we are therefore not convinced that the relationships between microfinance and outcomes are causal.

• Item Type: MPRA Paper
• Language: English
• Keywords: Microfinance; impact evaluation; Bangladesh; propensity score matching; sensitivity analysis
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C10 - General; C - Mathematical and Quantitative Methods > C3 - Multiple or Simultaneous Equation Models ; Multiple Variables > C31 - Cross-Sectional Models ; Spatial Models ; Treatment Effect Models ; Quantile Regressions ; Social Interaction Models; O - Economic Development, Innovation, Technological Change, and Growth > O1 - Economic Development > O12 - Microeconomic Analyses of Economic Development

References

Abou-Ali, H., El-Azony, H., El-Laithy, H., Haughton, J. and Khandker, S. R. (2009) Evaluating the Impact of Egyptian Social Fund for Development Programs. World Bank Policy Research Working Paper No. 4993, July.

Alexander, G. (2001) An Empirical Analysis of Microfinance: Who are the Clients? Northeastern Universities Development Consortium Conference. Boston, 28-30 September 2001.

Armendáriz de Aghion, B. and Morduch, J. (2005) The Economics of Microfinance. Cambridge: MIT Press.

Armendáriz de Aghion, B. and Morduch, J. (2010) The Economics of Microfinance. 2nd ed. Cambridge: MIT Press.

Banerjee, A., Duflo, E., Glennerster, R. and Kinnan, C. (2009). The Miracle of Microfinance? Evidence from a Randomized Evaluation. Available at: http://econ-www.mit.edu/files/4162.

Bangladesh Bureau of Statistics (1985) Statistical Yearbook of Bangladesh 1984-85. Dhaka: People's Republic of Bangladesh.

Bangladesh Bureau of Statistics (2004) Statistical Yearbook of Bangladesh 2004. Dhaka: People's Republic of Bangladesh.

Bateman, M. and Chang, H.-J. (2009) The Microfinance Illusion. Available at: http://www.econ.cam.ac.uk/faculty/chang/pubs/Microfinance.pdf.

Becker, S. O. and Caliendo, M. (2007) Sensitivity Analysis for Average Treatment Effects. The STATA Journal, 7 (1), pp.71-83.

Becker, S. O. and Ichino, A. (2002). Estimation of Average Treatment Effects Based on Propensity Scores. The STATA Journal, 2 (4), pp.358-377.

Caliendo, M. and Hujer, R. (2005) The Microeconometric Estimation of Treatment Effects - An Overview. Forschungsinstitut zur Zukunft der Arbeit (IZA) Discussion Paper No. 1653, July.

Caliendo, M. and Kopeinig, S. (2005) Some Practical Guidance for the Implementation of Propensity Score Matching. Forschungsinstitut zur Zukunft der Arbeit (IZA) Discussion Paper No. 1588, May.

Caliendo, M. and Kopeinig, S. (2008) Some Practical Guidance for the Implementation of Propensity Score Matching. Journal of Economic Surveys, 22 (1), pp.31-72.

Chemin, M. (2008) The Benefits and Costs of Microfinance: Evidence from Bangladesh. Journal of Development Studies, 44 (4), pp.463-484.

Coleman, B. E. (1999) The Impact of Group Lending in Northeast Thailand. Journal of Development Economics, 60 (1), pp.105-141.

Coleman, B. E. (2006) Microfinance in Northeast Thailand: Who Benefits and How Much? World Development, 34 (9), pp.1612-1638.

Cornfield, J., Haenszel, W., Hammond, E. and Lilienfeld, A. (1959) Smoking and Lung Cancer: Recent Evidence and a Discussion of Some Questions. Journal of the National Cancer Institute, 22, pp.173-203.

Deaton, A. (2009) Instruments of Development: Randomization in the Tropics, and the Search for the Elusive Keys to Economic Development. Available at: http://www.princeton.edu/~deaton/downloads/Instruments_of_Development.pdf.

Dichter, T. and Harper, M. eds. (2007) What's Wrong with Microfinance? Warwickshire: Practical Action Publishing.

DiPrete, T. A. and Gangl, M. (2004) Assessing Bias in the Estimation of Causal Effects: Rosenbaum Bounds on Matching Estimators and Instrumental Variables Estimation with Imperfect Instruments. Sociological Methodology, 34 (1), pp.271-310.

Fernando, J. L. (1997) Nongovernmental Organizations, Micro-Credit, and Empowerment of Women. The ANNALS of the American Academy of Political and Social Science, 554 (1), pp.150-177.

Gaile, G. L. and Foster, J. (1996) Review of Methodological Approaches to the Study of the Impact of Microenterprise Credit Programs. Report submitted to USAID Assessing the Impact of Microenterprise Services (AIMS), June.

Goetz, A. M. and Sen Gupta, R. (1996) Who Takes the Credit? Gender, Power, and Control Over Loan Use in Rural Credit Programs in Bangladesh. World Development, 24 (1), pp.45-63.

Goldberg, N. (2005) Measuring the Impact of Microfinance: Taking Stock of What We Know. Grameen Foundation USA Publication Series, December.

Hamermesh, D. S. (2007) Viewpoint: Replication in Economics. Canadian Journal of Economics, 40 (3), pp.715-733.

Heckman, J. J. (1979) Sample Selection Bias as a Specification Error. Econometrica, 47 (1), pp.153-161.

Heckman, J. J., Ichimura, H., Smith, J. and Todd, P. (1998) Characterizing Selection Bias Using Experimental Data. Econometrica, 66 (5), pp.1017-1098.

Heckman, J. J., Ichimura, H. and Todd, P. (1998) Matching as an Econometric Evaluation Estimator. The Review of Economic Studies, 65 (2), pp.261-294.

Hulme, D. and Mosley, P. (1996) Finance against Poverty. London: Routledge.

Ichino, A., Mealli, F. and Nannicini, T. (2006) From Temporary Help Jobs to Permanent Employment: What Can We Learn from Matching Estimators and their Sensitivity? Forschungsinstitut zur Zukunft der Arbeit (IZA) Discussion Paper No. 2149, May.

Imbens, G. (2009) Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009). NBER Working Paper No. 14896.

Karlan, D. and Zinman, J. (2009) Expanding Microenterprise Credit Access: Using Randomized Supply Decisions to Estimate the Impacts in Manila. Available at: http://karlan.yale.edu/p/expandingaccess_manila_jul09.pdf.

Khandker, S. R. (1996) Role of Targeted Credit in Rural Non-farm Growth. Bangladesh Development Studies, 24 (3 & 4).

Khandker, S. R. (1998) Fighting Poverty with Microcredit: Experience in Bangladesh. New York: Oxford University Press.

Khandker, S. R. (2000) Savings, Informal Borrowing and Microfinance. Bangladesh Development Studies, 26 (2 & 3).

Khandker, S. R. (2005) Microfinance and Poverty: Evidence Using Panel Data from Bangladesh. The World Bank Economic Review, 19 (2), pp.263-286.

Leamer, E. E. (1983) Let's Take the Con Out of Econometrics. The American Economic Review, 73 (1), pp.31-43.

Leuven, E. and Sianesi, B. (2003) PSMATCH2: STATA Module to Perform Full Mahalanobis and Propensity Score Matching, Common Support Graphing, and Covariate Imbalance Matching. Available at: http://ideas.repec.org/c/boc/bocode/s432001.html.

McKernan, S.-M. (2002) The Impact of Microcredit Programs on Self-Employment Profits: Do Noncredit Program Aspects Matter? Review of Economics and Statistics, 84 (1), pp.93-115.

Menon, N. (2006) Non-linearities in Returns to Participation in Grameen Bank Programs. Journal of Development Studies, 42 (8), pp.1379 - 1400.

Morduch, J. (1998) Does Microfinance Really Help the Poor? New Evidence from Flagship Programs in Bangladesh. Unpublished mimeo.

Morduch, J. (1999) The Microfinance Promise. Journal of Economic Literature, XXXVII December, pp.1569-1614.

Morduch, J. and Haley, B. (2002) Analysis of the Effects of Microfinance on Poverty Reduction. NYU Wagner Working Paper No. 1014, June.

Morgan, S. L. and Winship, C. (2007) Counterfactuals and Causal Inference. Methods and Principles for Social Research. Cambridge: Cambridge University Press.

Nannicini, T. (2007) Simulation-based Sensitivity Analysis for Matching Estimators. The STATA Journal, 7 (3), pp.334-350.

Neyman, J. S. (1923) On the Application of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9. Translated in Statistical Science, 5 (4), pp.465-480, 1990.

Odell, K. (2010) Measuring the Impact of Microfinance: Taking Another Look. Grameen Foundation USA Publication Series, May.

Pitt, M., Khandker, S. R. and Cartwright, J. (2006) Empowering Women with Micro-finance: Evidence from Bangladesh. Economic Development and Cultural Change, pp.791-831.

Pitt, M., Khandker, S. R., Chowdhury, O. H. and Millimet, D. L. (2003) Credit Programmes for the Poor and the Health Status of Children in Rural Bangladesh. International Economic Review, 44 (1), pp.87-118.

Pitt, M. M. (1999) Reply to Jonathan Morduch’s “Does Microfinance Really Help the Poor? New Evidence from Flagship Programs in Bangladesh”. Unpublished mimeo.

Pitt, M. M. (2000) The Effect of Nonagricultural Self-Employment Credit on Contractual Relations and Employment in Agriculture: The Case of Microcredit Programs in Bangladesh. Bangladesh Development Studies, 26 (2 & 3), pp.15-48.

Pitt, M. M. and Khandker, S. R. (1998) The Impact of Group-Based Credit Programs on Poor Households in Bangladesh: Does the Gender of Participants Matter? Journal of Political Economy, 106 (5), pp.958-996.

Pitt, M. M. and Khandker, S. R. (2002) Credit Programmes for the Poor and Seasonality in Rural Bangladesh. Journal of Development Studies, 39 (2), pp.1-24.

Pitt, M. M., Khandker, S. R., McKernan, S.-M. and Latif, M. A. (1999) Credit Programs for the Poor and Reproductive Behavior of Low-Income Countries: Are the Reported Causal Relationships the Result of Heterogeneity Bias? Demography, 36 (1), pp.1-21.

Pritchett, L. (2009) The Policy Irrelevance of the Economics of Education: Is "Normative as Positive" Just Useless, or Worse? In Cohen, J. and Easterly, W., eds. What Works in Development? Thinking Big and Thinking Small. Washington D.C.: Brookings Institution Press.

Ravallion, M. (2001) The Mystery of the Vanishing Benefits: An Introduction to Impact Evaluation. The World Bank Economic Review, 15 (1), pp.115-140.

Ravallion, M. (2008) Evaluating Anti-Poverty Programs. In Schultz, T. P. and Strauss, J., eds. Handbook of Development Economics, Volume 4. Amsterdam: Elsevier.

Roodman, D. and Morduch, J. (2009) The Impact of Microcredit on the Poor in Bangladesh: Revisiting the Evidence. Center for Global Development, Working Paper No. 174, June.

Rosenbaum, P. R. (2002) Observational Studies. New York: Springer.

Rosenbaum, P. R. and Rubin, D. B. (1983) The Central Role of the Propensity Score in Observational Studies for Causal Effects. Biometrika, 70 (1), pp.41-55.

Rosenbaum, P. R. and Rubin, D. B. (1984) Reducing Bias in Observational Studies Using Subclassification on the Propensity Score. Journal of the American Statistical Association, 79 (387), pp.516-524.

Rubin, D. B. (1973a) Matching to Remove Bias in Observational Studies. Biometrics, 29 (1), pp.159-183.

Rubin, D. B. (1973b) The Use of Matched Sampling and Regression Adjustment to Remove Bias in Observational Studies. Biometrics, 29 (1), pp.185-203.

Rubin, D. B. (1974) Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies. Journal of Educational Psychology, 66 (5), pp.688-701.

Rubin, D. B. (1977) Assignment to Treatment Group on the Basis of a Covariate. Journal of Educational Statistics, 2 (1), pp.1-26.

Rubin, D. B. (1978) Bayesian Inference for Causal Effects: The Role of Randomization. The Annals of Statistics, 6 (1), pp.34-58.

Rutherford, S. (2001) The Poor and Their Money. New Delhi: Oxford University Press.

Sebstad, J. and Chen, G. (1996) Overview of Studies on the Impact of Microenterprise Credit. Report submitted to USAID Assessing the Impact of Microenterprise Services (AIMS), June.

Smith, J. A. and Todd, P. (2005) Does Matching Overcome LaLonde's Critique of Nonexperimental Estimators? Journal of Econometrics, 125, pp.305-353.

Venkata, N. A. and Yamini, V. (2010) Why Do Microfinance Clients Take Multiple Loans? MicroSave India Focus Note 33, February.

Yunus, M. (1999) Banker to the Poor: Micro-Lending and the Battle Against World Poverty. New York: PublicAffairs.