Smoke and Mirrors: Evidence of Microfinance Impact from an Evaluation of SEWA Bank in India

Duvendack, Maren (2010): Smoke and Mirrors: Evidence of Microfinance Impact from an Evaluation of SEWA Bank in India.

Abstract

Microfinance has been on the development agenda for more than 30 years, heralded as the wondrous tool that reduces poverty and empowers women (Hulme and Mosley, 1996; Rutherford, 2001; Morduch and Haley, 2002; Khandker, 1998). Doubts, however, have recently been raised about the success of microfinance (Dichter and Harper, 2007; Banerjee et al, 2009; Roodman and Morduch, 2009; Karlan and Zinman, 2009; Bateman and Chang, 2009). Given this context, this paper re-examines the microfinance impact evaluation of SEWA Bank conducted by the United States Agency for International Development (USAID) in India in 1998 and 2000. The USAID panel and a new cross-section data set are analysed using propensity score matching (PSM) and panel data techniques to address selection bias. Sensitivity analysis of the matching results is used to explore their reliability. Various sub-group comparisons between borrowers, savers and controls are also conducted to shed some light on the impact of savings versus credit. The paper concludes that doubts remain about the quality of the impact estimates obtained through advanced econometric techniques. Direct observation and the outcome of sensitivity analysis of the PSM analysis suggest that the application of PSM and differences-in-differences (DID) to these observational data were probably unable to account for selection on unobservables.

• Item Type: MPRA Paper
• Language: English
• Keywords: Impact evaluation, evaluation methods, selection bias, microfinance, India
• Subjects: O - Economic Development, Innovation, Technological Change, and Growth > O1 - Economic Development; O - Economic Development, Innovation, Technological Change, and Growth > O1 - Economic Development > O16 - Financial Markets ; Saving and Capital Investment ; Corporate Finance and Governance; C - Mathematical and Quantitative Methods > C0 - General > C01 - Econometrics

References

Adams, D. W., 1978. Mobilizing Household Savings through Rural Financial Markets. Economic Development and Cultural Change, 26 (3), p.547-560.

Aportela, F., 1999. Effects of Financial Access on Savings by Low-Income People. Available at: http://www.lacea.org/meeting2000/FernandoAportela.pdf.

Armendáriz de Aghion, B. & Morduch, J., 2005. The Economics of Microfinance. Cambridge: MIT Press.

Arun, T., Imai, K. & Sinha, F., 2006. Does the Microfinance Reduce Poverty in India? Propensity Score Matching based on a National-Level Household Data. Economics Discussion Paper, The University of Manchester, September.

Ashraf, N., Karlan, D. S. & Yin, W., 2006. Female Empowerment: Impact of a Commitment Savings Product in the Philippines. Available at: http://www.econ.yale.edu/growth_pdf/cdp949.pdf.

Augsburg, B., 2006. Econometric Evaluation of the SEWA Bank in India: Applying Matching Techniques based on the Propensity Score. Working Paper MGSoG/2006/WP003, Maastricht University, October.

Banerjee, A., Duflo, E., Glennerster, R. & Kinnan, C., 2009. The Miracle of Microfinance? Evidence from a Randomized Evaluation. Available at: http://econ-www.mit.edu/files/4162.

Basu, P., 2006. Improving Access to Finance for India's Rural Poor. Washington D.C.: The World Bank. Bateman, M. & Chang, H.-J., 2009. The Microfinance Illusion. Available at: http://www.econ.cam.ac.uk/faculty/chang/pubs/Microfinance.pdf.

Becker, S. O. & Caliendo, M., 2007. Sensitivity Analysis for Average Treatment Effects. The STATA Journal, 7 (1), p.71-83.

Browning, M. & Lusardi, A., 1996. Household Saving: Micro Theories and Micro Facts. Journal of Economic Literature, 34 (4), p.1797-1855.

Bryson, A., Dorsett, R. & Purdon, S., 2002. The Use of Propensity Score Matching in the Evaluation of Active Labour Market Policies. Policy Studies Institute and National Centre for Social Research, Working Paper No. 4, Department for Work and Pensions.

Burgess, R. & Pande, R., 2005. Do Rural Banks Matter? Evidence from the Indian Social Banking Experiment. The American Economic Review, 95 (3), p.780-795.

Caliendo, M., 2006. Microeconometric Evaluation of Labour Market Policies. Berlin: Springer.

Caliendo, M. & Hujer, R., 2005. The Microeconometric Estimation of Treatment Effects - An Overview. Forschungsinstitut zur Zukunft der Arbeit (IZA) Discussion Paper No. 1653, July.

Caliendo, M. & Kopeinig, S., 2005. Some Practical Guidance for the Implementation of Propensity Score Matching. Forschungsinstitut zur Zukunft der Arbeit (IZA) Discussion Paper No. 1588, May.

Caliendo, M. & Kopeinig, S., 2008. Some Practical Guidance for the Implementation of Propensity Score Matching. Journal of Economic Surveys, 22 (1), p.31-72.

Census of India, 2001. District Census Handbook, Part XII A&B. Ahmedabad: Government of India.

Chemin, M., 2008. The Benefits and Costs of Microfinance: Evidence from Bangladesh. Journal of Development Studies, 44 (4), p.463-484.

Chen, M. A. & Snodgrass, D., 1999. An Assessment of the Impact of SEWA Bank in India: Baseline Findings. Report submitted to USAID Assessing the Impact of Microenterprise Services (AIMS), August.

Chen, M. A. & Snodgrass, D., 2001. Managing Resources, Activities, and Risk in Urban India: The Impact of SEWA Bank. Report submitted to USAID Assessing the Impact of Microenterprise Services (AIMS), September.

Coleman, B. E., 1999. The Impact of Group Lending in Northeast Thailand. Journal of Development Economics, 60 (1), p.105-141.

Collins, D., Morduch, J., Rutherford, S. & Ruthven, O., 2009. Portfolios of the Poor: How the World's Poor Live on $2 a Day. Princeton: Princeton University Press.

Cornfield, J., Haenszel, W., Hammond, E. & Lilienfeld, A., 1959. Smoking and Lung Cancer: Recent Evidence and a Discussion of Some Questions. Journal of the National Cancer Institute, 22, p.173-203.

Deaton, A., 2009. Instruments of Development: Randomization in the Tropics, and the Search for the Elusive Keys to Economic Development. Available at: http://www.princeton.edu/~deaton/downloads/Instruments_of_Development.pdf.

Dehejia, R. H. & Wahba, S., 1999. Causal Effects in Nonexperimental Studies: Reevaluating the Evaluation of Training Programs. Journal of the American Statistical Association, 94 (448), p.1053-1062.

Dehejia, R. & Wahba, S., 2002. Propensity Score-Matching Methods for Nonexperimental Causal Studies. The Review of Economic Studies, 84 (1), p.151-161.

Devaney, P. L., 2006. Microsavings Programs: Assessing Demand and Impact, A Critical Review of the Literature. Assessing the Impact of Innovation Grants in Financial Services, IRIS Center, June.

Dichter, T. & Harper, M. eds., 2007. What's Wrong with Microfinance? Warwickshire: Practical Action Publishing.

Dieckmann, R., 2007. Microfinance: An Emerging Investment Opportunity: Uniting Social Investment and Financial Returns. Report completed for Deutsche Bank Research, December.

DiPrete, T. A. & Gangl, M., 2004. Assessing Bias in the Estimation of Causal Effects: Rosenbaum Bounds on Matching Estimators and Instrumental Variables Estimation with Imperfect Instruments. Sociological Methodology, 34 (1), p.271-310.

Duflo, E., Glennerster, R. & Kremer, M., 2007. Using Randomization in Development Economics Research: A Toolkit. Centre for Economic Policy Research, Discussion Paper No. 6059, January.

Duflo, E. & Kremer, M., 2003. Use of Randomization in the Evaluation of Development Effectiveness. Unpublished mimeo.

Dupas, P. & Robinson, J., 2009. Savings Constraints and Microenterprise Development: Evidence from a Field Experiment in Kenya. NBER Working Paper No. w14693.

Fernando, J. L., 1997. Nongovernmental Organizations, Micro-Credit, and Empowerment of Women. The ANNALS of the American Academy of Political and Social Science, 554 (1), p.150-177.

Fisher, T. & Sriram, M. S., 2002. Beyond Micro-Credit: Putting Development Back into Micro-Finance. New Delhi: Vistaar Publications.

Gaile, G. L. & Foster, J., 1996. Review of Methodological Approaches to the Study of the Impact of Microenterprise Credit Programs. Report submitted to USAID Assessing the Impact of Microenterprise Services (AIMS), June.

Ghate, P., 2007. Consumer Protection in Indian Microfinance: Lessons from Andhra Pradesh and the Microfinance Bill. Economic and Political Weekly, 42 (13), p.1176-1184.

Grootaert, C. & Bastelaer, T. v. eds., 2002. Understanding and Measuring Social Capital: A Multidisciplinary Tool for Practitioners. Washington D.C.: The World Bank.

Hamermesh, D. S., 2007. Viewpoint: Replication in Economics. Canadian Journal of Economics, 40 (3), p.715-733.

Harriss, J. & de Renzio, P., 1997. "Missing Link" or Analytically Missing? The Concept of Social Capital. Journal of International Development, 9 (7), p.919-937.

Heckman, J. J., Ichimura, H., Smith, J. & Todd, P., 1998. Characterizing Selection Bias Using Experimental Data. Econometrica, 66 (5), p.1017-1098.

Heckman, J. J., Ichimura, H. & Todd, P., 1997. Matching as an Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme. Review of Economic Studies, 64, p.605-654.

Heckman, J. J. & Urzua, S., 2009. Comparing IV with Structural Models: What Simple IV Can and Cannot Identify. NBER Working Paper No. 14706.

Hulme, D., 2000. Impact Assessment Methodologies for Microfinance: Theory, Experience and Better Practice. World Development, 28 (1), p.79-98.

Hulme, D. & Mosley, P., 1996. Finance against Poverty. London: Routledge.

Ichino, A., Mealli, F. & Nannicini, T., 2006. From Temporary Help Jobs to Permanent Employment: What Can We Learn from Matching Estimators and their Sensitivity? Forschungsinstitut zur Zukunft der Arbeit (IZA) Discussion Paper No. 2149, May.

Imbens, G., 2009. Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009). NBER Working Paper No. 14896.

Ito, S., 2003. Microfinance and Social Capital: Does Social Capital Help Create Good Practice? Development in Practice, 13 (4), p.322-332.

Karlan, D. S. & Morduch, J., 2009. Access to Finance. Available at: http://karlan.yale.edu/p/HDE_June_11_2009_Access_to_Finance.pdf.

Karlan, D. S. & Zinman, J., 2009. Expanding Microenterprise Credit Access: Using Randomized Supply Decisions to Estimate the Impacts in Manila. Available at: http://karlan.yale.edu/p/expandingaccess_manila_jul09.pdf.

Keynes, J. M., 1936. The General Theory of Employment, Interest and Money. London: Macmillan.

Khandker, S. R., 1998. Fighting Poverty with Microcredit: Experience in Bangladesh. New York: Oxford University Press.

Ledgerwood, J., 1999. Microfinance Handbook: An Institutional and Financial Perspective. Washington D.C.: The World Bank.

Levitt, S. D. & List, J. A., 2009. Was there Really a Hawthorne Effect at the Hawthorne Plant? An Analysis of the Original Illumination Experiments. NBER Working Paper No. 15016.

Manski, C. F., 1995. Identification Problems in the Social Sciences. Cambridge: Harvard University Press.

Miguel, E. & Kremer, M., 2004. Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities. Econometrica, 72 (1), p.159-217.

Molyneux, M., 2002. Gender and the Silences of Social Capital: Lessons from Latin America. Development and Change, 33 (2), p.167-188.

Morduch, J. & Haley, B., 2002. Analysis of the Effects of Microfinance on Poverty Reduction. NYU Wagner Working Paper No. 1014, June.

Morgan, S. L. & Harding, D. J., 2006. Matching Estimators of Causal Effects. Propects and Pitfalls in Theory and Practice. Sociological Methods & Research, 35 (1), p.3-60.

Morgan, S. L. & Winship, C., 2007. Counterfactuals and Causal Inference. Methods and Principles for Social Research. Cambridge: Cambridge University Press.

Pitt, M. M. & Khandker, S. R., 1998. The Impact of Group-Based Credit Programs on Poor Households in Bangladesh: Does the Gender of Participants Matter? Journal of Political Economy, 106 (5), p.958-996.

Pritchett, L., 2009. The Policy Irrelevance of the Economics of Education: Is "Normative as Positive" Just Useless, or Worse? In Cohen, J. & Easterly, W., eds. What Works in Development? Thinking Big and Thinking Small. Washington D.C.: Brookings Institution Press.

Rogg, C. S., 2000. The Impact of Access to Credit on the Saving Behavior of Microentrepreneurs: Evidence from 3 Latin American Countries. Available at: http://idbdocs.iadb.org/wsdocs/getdocument.aspx?docnum=1481486.

Roodman, D. & Morduch, J., 2009. The Impact of Microcredit on the Poor in Bangladesh: Revisiting the Evidence. Center for Global Development, Working Paper No. 174, June. Rosenbaum, P. R., 2002. Observational Studies. New York: Springer.

Rosenbaum, P. R., 2010. Design of Observational Studies. New York: Springer.

Rosenbaum, P. R. & Silber, J. H., 2001. Matching and Thick Description in an Observational Study of Mortality After Surgery. Biostatistics, 2 (2), p.217-232.

Rosenzweig, M. R., 2001. Savings Behaviour in Low-Income Countries. Oxford Review of Economic Policy, 17 (1), p.40-54.

Rutherford, S., 2001. The Poor and Their Money. New Delhi: Oxford University Press.

Saretsky, G., 1975. The John Henry Effect: Potential Confounder of Experimental vs Control Group Approaches to the Evaluation of Educational Innovations. The American Educational Research Association's Annual Meeting. Washington, D.C., 2 April 1975.

Sebstad, J. & Chen, G., 1996. Overview of Studies on the Impact of Microenterprise Credit. Report submitted to USAID Assessing the Impact of Microenterprise Services (AIMS), June.

Sebstad, J., Neill, C., Barnes, C. & Chen, G., 1995. Assessing the Impacts of Microenterprise Interventions: A Framework for Analysis. Center for Development Information and Evaluation, Working Paper No. 7, USAID, March.

Setboonsarng, S. & Parpiev, Z., 2008. Microfinance and the Millennium Development Goals in Pakistan: Impact Assessment Using Propensity Score Matching. Asian Development Bank Institute (ADBI) Discussion Paper No. 104, March.

Smith, J. A. & Todd, P., 2005. Does Matching Overcome LaLonde's Critique of Nonexperimental Estimators? Journal of Econometrics, 125, p.305-353.

Snodgrass, D. & Sebstad, J., 2002. Clients in Context: The Impacts of Microfinance in Three Countries: Synthesis Report. Report submitted to USAID Assessing the Impact of Microenterprise Services (AIMS), January.

Todd, H., 1996. Women at the Center: Grameen Bank Borrowers After One Decade. Boulder: Westview Press.

von Pischke, J. D., 1983. Towards an Operational Approach to Savings for Rural Developers. In Von Pischke, J. D., Adams, D. W. & Donald, G., eds. Rural Financial Markets in Developing Countries. Baltimore: Johns Hopkins University Press.