Methodological challenges in building composite indexes: Linking theory to practice

Santeramo, Fabio Gaetano (2016): Methodological challenges in building composite indexes: Linking theory to practice.

Abstract

Composite indicators are emerging in several fields and disciplines as appealing method to synthesize a multitude of information, in a compact, single, and unique way. The process of aggregating heterogeneous information is itself very challenging and exposed to numerous threats. The chapter deepens on the methodological challenges that scientists, analysts, and final users must be aware of for a correct interpretation of the composite indexes. By mean of a worked example on the construction of composite indicators for food security, the chapter concludes that while different normalization and weighting approaches do not alter composite indicators, data imputation and aggregation methods are the most crucial steps: different methods convey very different results. For instance, the adoption of different aggregation procedures may largely alter the rankings based on composite indicators. In sum, the analysis shows that the index construction decisions matter and comment on policy and practical implications for the construction of composite indicators.

• Item Type: MPRA Paper
• Language: English
• Keywords: Composite Indicator; Food Security; Data Aggregation; Data Imputation; Normalization; Weighting; FAO; Development; Economics
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C18 - Methodological Issues: General; C - Mathematical and Quantitative Methods > C8 - Data Collection and Data Estimation Methodology ; Computer Programs > C81 - Methodology for Collecting, Estimating, and Organizing Microeconomic Data ; Data Access; F - International Economics > F6 - Economic Impacts of Globalization > F63 - Economic Development; Q - Agricultural and Natural Resource Economics ; Environmental and Ecological Economics > Q1 - Agriculture; Q - Agricultural and Natural Resource Economics ; Environmental and Ecological Economics > Q1 - Agriculture > Q19 - Other

References

Alam, M., Dupras, J., & Messier, C. (2016). A framework towards a composite indicator for urban ecosystem services. Ecological Indicators, 60, 38-44.

Ames, A., Ames, A., Houston, J., & Angioloni, S. (2016). Does Financial Literacy Contribute to Food Security?. International Journal of Food and Agriculture Economics. 4, 21-34.

Aurino, E. (2014). Selecting a Core Set of indicators for Monitoring Global Food Security. A Methodological Proposal. FAO Statistics Division Working Papers 14-06, April.

Booysen, F. (2002). An overview and evaluation of composite indices of development. Social indicators research, 59(2), 115-151.

Carletto, C., Zezza, A., & Banerjee, R. (2013). Towards better measurement of household food security: Harmonizing indicators and the role of household surveys. Global Food Security, 2(1), 30-40.

Cafiero, C., H. R. Melgar‐Quiñonez, T. J. Ballard & A.W. Kepple. (2014). Validity and reliability of food security measures. Annals of the New York Academy of Sciences, 1331 (1), 230-248.

Cafiero, C. (2013). What do we really know about food security? (No. w18861). National Bureau of Economic Research.

Caracciolo, F., & Santeramo, F. G. (2013). Price Trends and Income Inequalities: Will Sub‐Saharan Africa Reduce the Gap?. African Development Review, 25(1), 42-54.

Carman, K. G., & Zamarro, G. (2016). Does Financial Literacy Contribute to Food Security?. International Journal of Food and Agriculture Economics, 4, 1-19.

Dobrota, M., Martic, M., Bulajic, M., & Jeremic, V. (2015). Two-phased composite I-distance indicator approach for evaluation of countries’ information development. Telecommunications Policy, 39(5), 406-420.

Economist Intelligence Unit (2013). The Global Food Security Index 2013: An annual measure of the state of global food security.

Food and Agricultural Organization (2013). New metrics to measure and monitor performance in agriculture and food security, held on 5 December 2012 in Rome.

Freudenberg, M. (2003). Composite indicators of country performance: a critical assessment", OECD, Paris.

Gabbert, S., & Weikard, H. (2001). How widespread is undernourishment? A critique of measurement methods and new empirical results. Food Policy, 26, 209-228.

Godfray, H. C. J., Beddington, J. R., Crute, I. R., Haddad, L., Lawrence, D., Muir, J. F., Jules Pretty, Sherman Robinson, Sandy M. Thomas, & Toulmin, C. (2010). Food security: the challenge of feeding 9 billion people. Science, 327(5967), 812-818.

Jacobs, R. P. Smith, & M. Goddard (2004). Measuring performance: an examination of composite performance indicators. Centre for Health Economics, Technical Paper Series 29.

Hertel, T. W. (2016). Food security under climate change. Nature Climate Change, 6(1), 10-13.

IFPRI (2011). Global hunger index – the challenge of hunger: price spikes and excessive food price volatility. Bonn, Washington, DC, Dublin.

Leys, C., Ley, C., Klein, O., Bernard, P., & Licata, L. (2013). Detecting outliers: do not use standard deviation around the mean, use absolute deviation around the median. Journal of Experimental Social Psychology, 49(4), 764-766.

Mahadevan, R., & Hoang, V. (2015). Is There a Link Between Poverty and Food Security?. Social Indicators Research, 1-21.

Malinowski, E. R. (2009). Determination of rank by median absolute deviation (DRMAD): a simple method for determining the number of principal factors responsible for a data matrix. Journal of Chemometrics, 23(1), 1-6.

Maricic, M., Bulajic, M., Dobrota, M., & Jeremic, V. (2016) Redesigning the Global Food Security Index: A Multivariate Composite I-Distance Indicator Approach. International journal of food and agricultural economics, 4(1), 69-86.

Munda G., 2012. Choosing aggregation rules for composite indicators. Social Indicator Research, 109(3), 337-354.

Munda, G., & Nardo, M. (2003). On the methodological foundations of composite indicators used for ranking countries. Ispra, Italy: Joint Research Centre of the European Communities.

Munda, G., & Saisana, M. (2011). Methodological considerations on regional sustainability assessment based on multicriteria and sensitivity analysis. Regional Studies, 45(2), 261-276.

Nardo, M., Saisana, M., Saltelli, A., Tarantola, S., Hoffman, A., & Giovannini, E. (2005a). Handbook on constructing composite indicators: methodology and user guide (No. 2005/3). OECD publishing.

Nardo, M., Saisana, M., Saltelli, A., & Tarantola, S. (2005b). Tools for composite indicators building. European Comission, Ispra.

Ouertani, E. (2016). Food Security in Tunisia Within Water Scarcity the Relative Importance of The Meat Sector. International journal of food and agricultural economics, 4(1), 35-54.

Paruolo, P., Saisana, M., & Saltelli, A. (2013). Ratings and rankings: voodoo or science?. Journal of the Royal Statistical Society: Series A, 176(3), 609-634.

Pemberton, C., Patterson-Andrews, H., & De Sormeaux (2016) Relative Vulnerability of Selected Caribbean States to Changes in Food Security Due To Tropical Storms and Hurricanes. International journal of food and agricultural economics, 4(1), 125-136.

Pinstrup-Andersen P. (2009) Food security: definition and measurement. Food Security, 1, 5–7.

Saaty, T. L. (2001). Decision Making for Leaders: The Analytic Hierarchy Process for Decisions in a Complex World. 1999/2000 Edition (Vol. 2). RWS publications.

Saisana, M., & Saltelli, A. (2013). Joint Research Centre Statistical Audit of the 2014 Global Innovation Index. The Global Innovation Index 2014, 55.

Santeramo, F. G., Di Pasquale, J., Contò, F., Tudisca, S., & Sgroi, F. (2012). Analyzing risk management in Mediterranean Countries: the Syrian perspective. New Medit, 11(3), 35-40.

Santeramo, F. G. (2014). On the composite indicators for food security: Decisions matter!. MPRA Paper.

Santeramo, F.G. (2015). Indicatori compositi di Food Security: quali implicazioni per i policymaker? AgriregioniEuropa, 41, 102-104.

Santeramo F.G. (2015) On the composite indicators for food security: decision matter! Food Reviews International. 1, 63-73.

Santeramo, F. G. (2015). Food security composite indices: implications for policy and practice. Development in Practice, 25(4), 594-600.

Santeramo, F. G., & Shabnam, N. (2015). The income-elasticity of calories, macro-and micro-nutrients: What is the literature telling us?. Food Research International, 76, 932-937.

Santeramo, F.G. (2016). Il consumo di calorie, micro e macro nutrienti: cosa insegnano le elasticità al reddito? AgriregioniEuropa, 44.

Svedberg, P., 2011. How Many People are Malnourished? Annual Review of Nutrition, 31, 263-283.

Tate, E. (2012). Social vulnerability indices: a comparative assessment using uncertainty and sensitivity analysis. Natural Hazards, 63(2), 325-347.

Wheeler, T., & von Braun, J. (2013). Climate change impacts on global food security. Science, 341(6145), 508-513.

Zornic, N., Bornmann, L., Maricic, M., Markovic, A., Martic, M., & Jeremic, V. (2015). Ranking Institutions within a University based on their scientific performance: a percentile-based approach. El profesional de la información, 24(5).