Why inferential statistics are inappropriate for development studies and how the same data can be better used

Ballinger, Clint (2011): Why inferential statistics are inappropriate for development studies and how the same data can be better used.

Abstract

The purpose of this paper is twofold:

1) to highlight the widely ignored but fundamental problem of ‘superpopulations’ for the use of inferential statistics in development studies. We do not to dwell on this problem however as it has been sufficiently discussed in older papers by statisticians that social scientists have nevertheless long chosen to ignore; the interested reader can turn to those for greater detail.

2) to show that descriptive statistics both avoid the problem of superpopulations and can be a powerful tool when used correctly. A few examples are provided.

The paper ends with considerations of some reasons we think are behind the adherence to methods that are known to be inapplicable to many of the types of questions asked in development studies yet still widely practiced.

• Item Type: MPRA Paper
• Language: English
• Keywords: frequentist statistics; Bayesian statistics; causation; determinism; explanation; spatial autocorrelation; mulitple regression; international development; econometrics; comparative method; datasets; descriptive statistics; tabular analysis; visual analysis; maps; regession modeling; quantitative; qualitative; macrosociology; superpopulations; apparent populations; indeterminism; statistical assumptions
• Subjects: B - History of Economic Thought, Methodology, and Heterodox Approaches > B0 - General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C12 - Hypothesis Testing: General; C - Mathematical and Quantitative Methods > C3 - Multiple or Simultaneous Equation Models ; Multiple Variables > C33 - Panel Data Models ; Spatio-temporal Models; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C11 - Bayesian Analysis: General; P - Economic Systems > P1 - Capitalist Systems > P16 - Political Economy; A - General Economics and Teaching > A1 - General Economics > A11 - Role of Economics ; Role of Economists ; Market for Economists; O - Economic Development, Innovation, Technological Change, and Growth > O1 - Economic Development; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C10 - General; F - International Economics > F5 - International Relations, National Security, and International Political Economy; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C20 - General; C - Mathematical and Quantitative Methods > C3 - Multiple or Simultaneous Equation Models ; Multiple Variables; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C23 - Panel Data Models ; Spatio-temporal Models; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C21 - Cross-Sectional Models ; Spatial Models ; Treatment Effect Models ; Quantile Regressions

References

Bairoch, Paul. 1985/1988. Cities and Economic Development. Translated by Christopher Braider. Chicago: University of Chicago Press.

Bairoch, Paul. 1993. Economics and World History: Myths and Paradoxes. London: Harvester Wheatsheaf.

Ballinger, Clint. 2008. Initial Conditions as Exogenous Factors in Spatial Explanation. Ph.D. Dissertation, Department of Geography, University of Cambridge. Available online at: http://philosophyofscience.webstarts.com/working_papers.html

Ballinger, Clint. 2011a. Comparative Economics in a World Divided: Spatial Autocorrelation and World Regions. http://philosophyofscience.webstarts.com/working_papers.html

Ballinger, Clint. 2011b. Acemoglu, Johnson, and Robinson: Natural Experiments or Geographic Theories of Development? http://philosophyofscience.webstarts.com/working_papers.html

Ballinger, Clint. 2011c. Two Fatal Flaws in the ‘Reversal of Fortune’ (Acemoglu, Johnson, and Robinson 2002) Argument. http://philosophyofscience.webstarts.com/working_papers.html

Ballinger, Clint. 2011d. Mercantilism and Uneven Development. http://philosophyofscience.webstarts.com/working_papers.html

Ballinger, Clint. 2011e. World Regions, International Development and Cluster Analysis. http://philosophyofscience.webstarts.com/working_papers.html

Berk, Richard A., Bruce Western and Robert E. Weiss. 1995. Statistical Inference for Apparent Populations. Sociological Methodology 25: 421-458.

Berk, Richard A., Bruce Western and Robert E. Weiss. 1995. Reply to Bollen, Firebaugh, and Rubin. Sociological Methodology 25: 481-485.

Berk, Richard A. and David Freedman. 2001. ‘Statistical Assumptions as Empirical Commitments’. UCLA Department of Statistics Papers, Paper 2001080101.

Bertin, J. 1983. Semiology of Graphics. Madison: University of Wisconsin Press.

Bollen, Kenneth A, Barbara Entwisle and Arthur S Alderson. 1993. Macrocomparative research methods. Annual Review of Sociology 19: 321-51.

Caplan, Bryan. 2003. ‘Why I am not an Austrian Economist’. Unpublished manuscript, Department of Economics, George Mason University. Accessed January 2011. http://www.econfaculty/gmu.edu/bcaplan/whyaust.htm

Cleveland, William S. 1993. Visualizing Data., Summit, NJ: Hobart Press.

Cleveland, William S. 1994. The Elements of Graphing Data. Summit, NJ: Hobart Press.

Darmofal, David. 2006. ‘Spatial Econometrics and Political Science’. Paper presented at the Annual Meeting of the Southern Political Science Association, Atlanta Georgia, January 5-7, 2006.

Efron, B. 1986. Why isn't everyone a Bayesian? The American Statistician 40: 1-11.

Firebaugh, Glenn. 1995. Will Bayesian Inference Help? A Skeptical View. Sociological Methodology 25: 469-472.

Freedman, David. A. 1991a. Statistical Models and Shoe Leather. Sociological Methodology 21: 291-313.

Freedman, David A. 1991b. A Rejoinder to Berk, Blalock, and Mason. Sociological Methodology 21: 353-358.

Georgescu-Roegen, Nicholas. 1966. Analytical Economics: Issues and Problems. Cambridge, Mass: Harvard University Press.

Goertzel, Ted. 2002. ‘Econometric Modeling as Junk Science’ unpublished manuscript, Department of Sociology, Rutgers University. (A simplified version without graphs is published as ‘Myths of Murder and Multiple Regression’, The Skeptical Inquirer. 26(1): 19-23; the graphs, however, are important.)

Goertzel, Ted and Benjamin Goertzel, 2008. Capital Punishment and Homicide Rates: Sociological Realities and Econometric Distortions. Critical Sociology 34(2). 239-254.

Goodchild, Michael F. 2009. What Problem? Spatial autocorrelation and geographic information science. Geographical Analysis 41(4): 411-417.

Graeber, David. 2004. Fragments of an Anarchist Anthropology. Chicago: Prickly Paradigm Press.

Hall, Peter A. and Robert J. Franzese. 1998. Mixed Signals: Central Bank Independence, Coordinated Wage Bargaining, and European Monetary Union. International Organization 52: 505-535.

Haining, Robert. 2003. Spatial Data Analysis: Theory and Practice. Cambridge: Cambridge University Press.

Hamnett, Chris. 2003. Contemporary Human Geography: Fiddling While Rome Burns? Geoforum 34: 1-3.

Hoover, Kevin D. 2001. Causality in Macroeconomics. Cambridge: Cambridge University Press.

Kittel, Bernhard. 2006. A Crazy Methodology? On the Limits of Macro-Quantitative Social Science Research. International Sociology 21: 647-677.

Köbben, A. J. F. 1967. Why Exceptions? The Logic of Cross-Cultural Analysis. Current Anthropology 8(1-2): 3-34.

Kundera, Milan. 1984.  The Unbearable Lightness of Being. London: Faber and Faber.

Markusen, Anne. 1999. Fuzzy Concepts, Scanty Evidence, Policy Distance: The Case for Rigour and Policy Relevance in Critical Regional Studies. Regional Studies 33: 869-884.

Martin, R. L. 2000. Editorial: In memory of maps. Transactions of the Institute of British Geographers 25: 3-5.

Martin, Ron and Peter Sunley. 2001. Rethinking the ‘Economic’ in Economic Geography: Broadening Our Vision or Losing Our Focus? Antipode 33: 149-161.

Pearl, Judea. 2003. Statistics and Causal Inference: A Review. Sociedad de Estadística e Investigación Operativa (Test) 12(2): 281–345.

Orford, Scott, Richard Harris, and Daniel Dorling. 1999. Geography: Information Visualization in the Social Sciences, A State-of-the-Art Review. Social Science Computer Review 17: 289-304.

Povoledo, Elisabetta. 2007. In Italy, Creating Worlds Takes Precision, Yes, and Politics. The New York Times. August 15.

Rogowski, Ronald. 1995. The Role of Theory and Anomaly in Social Science Inference. The American Political Science Review 89:467-470.

Rothstein, Bo. 1990. ‘Labour Market Institutions and Working Class Strength’, in Structuring Politics: Historical Institutionalism in Comparative Analysis, Steinmo, S., K. Thelen and F. Longsteth, eds. Cambridge: Cambridge University Press.

Ross, Mark Howard and Elizabeth Homer. 1976. Galton’s Problem in Cross-National Research. World Politics 29(1): 1-28.

Shalev, Michael. 1998. ‘Limits of and Alternatives to Multiple Regression in Macro-Comparative Research’, presented at the 2nd conference on The Welfare State at the Crossroads, Stockholm, June 12-14.

Shalev, Michael. 2007. Limits and Alternatives to Multiple Regression in Comparative Research. Comparative Social Research. 24: 267-314.

Tufte, Edward R. 1997. Visual Explanations: Images and Quantities, Evidence and Narrative. Cheshire, Connecticut: Graphics Press.

Udy, Stanley H, Jr. 1973. Cross-Cultural Analysis: Methods and Scope. Annual Review of Anthropology 2: 253-270.

Western, Bruce and Simon Jackman. 1994. Bayesian Inference for Comparative Research. The American Political Science Review 88(2): 412-423.

Wilson, Edward O. 1998. Consilience: The Unity of Knowledge. London: Little, Brown and Company.