Using Deep Learning Neural Networks to Predict the Knowledge Economy Index for Developing and Emerging Economies

Andres, Antonio Rodriguez and Otero, Abraham and Amavilah, Voxi Heinrich (2021): Using Deep Learning Neural Networks to Predict the Knowledge Economy Index for Developing and Emerging Economies. Published in: Expert Systems with Applications , Vol. 184, No. https://doi.org/10.1016/j.eswa.2021.115514 (1 December 2021)

Abstract

Missing values and the inconsistency of the measures of the knowledge economy remain vexing problems that hamper policy-making and future research in developing and emerging economies. This paper contributes to the new and evolving literature that seeks to advance better understanding of the importance of the knowledge economy for policy and further research in developing and emerging economies. In this paper we use a supervised machine deep learning neural network (DLNN) approach to predict the knowledge economy index of 71 developing and emerging economies during the 1995-2017 period. Applied in combination with a data imputation procedure based on the K-closest neighbor algorithm, DLNN is capable of handling missing data problems better than alternative methods. A 10-fold validation of the DLNN yielded low quadratic and absolute error (0,382 +- 0,065). The results are robust and efficient, and the model’s predictive power is high. There is a difference in the predictive power when we disaggregate countries in all emerging economies versus emerging Central European countries. We explain this result and leave the rest to future endeavors. Overall, this research has filled in gaps due to missing data thereby allowing for effective policy strategies. At the aggregate level development agencies, including the World Bank that originated the KEI, would benefit from our approach until substitutes come along.

• Item Type: MPRA Paper
• Language: English
• Keywords: Machine deep learning neural networks; developing economies, emerging economies, knowledge economy, knowledge economy index, World Bank
• Subjects: C - Mathematical and Quantitative Methods > C4 - Econometric and Statistical Methods: Special Topics > C45 - Neural Networks and Related Topics; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C53 - Forecasting and Prediction Methods ; Simulation Methods; O - Economic Development, Innovation, Technological Change, and Growth > O3 - Innovation ; Research and Development ; Technological Change ; Intellectual Property Rights > O38 - Government Policy; O - Economic Development, Innovation, Technological Change, and Growth > O4 - Economic Growth and Aggregate Productivity > O41 - One, Two, and Multisector Growth Models; O - Economic Development, Innovation, Technological Change, and Growth > O5 - Economywide Country Studies > O57 - Comparative Studies of Countries; P - Economic Systems > P4 - Other Economic Systems > P41 - Planning, Coordination, and Reform

References

Ahmadi, M., Jafarzadeh-Ghoushchi, S., Taghizadeh, R., and Sharifi, A. (2019). Presentation of a new hybrid approach for forecasting economic growth using artificial intelligence approaches. Neural Computing and Applications, 31(12), 8661-8680.

Albawi, S., Mohammed, T. A., and Al-Zawi, S. (2017, August). Understanding of a convolutional neural network. In 2017 International Conference on Engineering and Technology (ICET) (pp. 1-6).

Albeaik, S., Kaltenberg, M., Alsaleh, M., & Hidalgo, C.A. (2017). Improving the economic complexity index. Cornell University. https://arxiv.org/ftp/arxiv/papers/1707/1707.05826.pdf. Accessed May 15, 2019.

Allison, P.D. (2001). Missing data. Thousand Oaks (CA): Sage.

Al Shami, A., Lotfi, A., Coleman, S., and Dostál, P. (2015). Unified knowledge based economy hybrid forecasting. Technological Forecasting and Social Change, 91, 107-123.

Al Shami, A., Lotfi, A., Lai, E., & Coleman, S. (2011, April). Unified knowledge economy competitiveness index using fuzzy clustering model. In 2011 IEEE Symposium on Computational Intelligence for Financial Engineering and Economics (CIFEr) (pp. 1-6). IEEE

Archibugi, D., & Coco, A. (2004). A new indicator of technological capabilities for developed and developing countries (ArCo). World Development, 32(4): 629-654.

Archibugi, D., & Coco, A. (2005). Measuring technological capabilities at the country level: A survey and a menu for choice. Research Policy, 34(2): 175-194.

Arrow, K.J. (1969). Classificatory notes on the production and transmission of technological knowledge. American Economic Review, 59(2): 29-35.

Ashwell, K.W. (2012). The brain book: Development, Function, Disorder, Health (pp. 230-233). New York: Firefly Books.

Athey, S. (2017). Beyond prediction: Using big data for policy problems. Science 355(6324): 483–485.

Athey, S. (2019). The Impact of Machine Learning on Economics. The Economics of Artificial Intelligence: An Agenda. Ajay Agrawal, Joshua Gans, and Avi Goldfarb, editors (p. 507 -547). Conference held September 13-14, 2017.

Athey, S., & Imbens, G.W. (2019). Machine learning methods economists should know about. Available at https://arxiv.org/abs/1903.10075. Accessed June 8, 2019.

Barlow, H.B. (1989). Unsupervised learning. Neural computation, 1(3): 295-311.

Becker, J., & Prischmann, M. (1993). Supporting the design process with neural networks. A complex application of cooperating neural networks and its implementation Journal of Information Science Technology, 3 (1): 79-95

Bishop, C.M. (2006). Pattern Recognition and Machine Learning. Springer.

Blackburn, K., & Forgues-Puccio, G.F. (2010). Financial liberalization, bureaucratic corruption, and economic development. Journal of International Money and Finance, 29(7): 1321–1339.

Breschi, S., & Lissoni, F. (2004). Chapter 28 Knowledge Networks from Patent Data: Methodological Issues and Research Targets. In Handbook of Quantitative Science and Technology Research The use of Publication and Patent statistics in Studies of S & T Systems. Edited by Henk F. Moed, Wolfgang Glänzel, and Ulrich Schmoch. Chapter 28.

Caudill, M. (1987). Neural networks primer, Part 1. AI Expert 2 (12): 46-52.

Chalfin, A., Danieli, O., Hillis, A., Jelveh, Z., Luca, M., Ludwig, J., & and Mullainathan, S. (2016). Productivity and selection of human capital with machine learning, American Economic Review, 106(5): 124–127.

Chen, D.H.C., & Dahlman, C.J. (2006). The Knowledge Economy, The KAM Methodology and World Bank Operations. http://documents.worldbank.org/curated/en/695211468153873436/pdf/358670WBI0The11dge1Economy01PUBLIC1.pdf . Accessed May 15, 2019.

Clevert, D.A., Unterthiner, T., and Hochreiter, S. (2015). Fast and accurate deep network learning by exponential linear units (elus). arXiv preprint arXiv:1511.07289.

Cook, D. (2016). Practical Machine Learning with H2O: Powerful, Scalable Techniques for Deep Learning and AI. O'Reilly Media, Inc.

De la Paz-Marín, M., Campoy-Muñoz, P., & Martínez, C.H. (2012). Non-linear multi-classifier model based on Artificial Intelligence to predict research and development performance in European countries, Technological Forecasting and Social Change, 79(9): 1731-1745.

De la Paz-Marín, M., Gutiérrez, P.A, Martínez, C.H. (2015). Classification of countries’ progress toward a knowledge economy based on machine learning classification techniques. Expert Systems with Applications, 42(1): 562-572.

Dodgson, M., & Gann, D. (2018). Innovation: A very short introduction. Oxford: Oxford University Press.

Efron, B. (1994). Missing data. Imputation, and bootstrap. Journal of the American Statistical Association, 89(426): 463-475.

Furman, J.L., Porter, M.E., & Stern, S. (2002). The determinants of national innovative capacity. Research Policy 31(6): 899-933.

Gelebo, E., Plekhanov, A., and Silve, F. (2015). Determinants of frontier innovation and technology adoption: cross-country evidence. Working Paper No. 173. European Bank for Reconstruction and Development (EBRD).

Gernert, D. (1998). Classification and prediction by neural networks, and their social implications. In Computer Modeling of Social process. Edited by Win B.G. Liebrand, Andrzej Nowak and Rainer Hegselman. London: Sage Publications. Chapter 5.

Gershonson, C. (2003). Artificial neural networks for beginners. https://arxiv.org/ftp/cs/papers/0308/0308031.pdf. Accessed June 18, 2019.

Griffiths, M.D., & Kickul, J. (2008). The socioeconomic determinants of innovation: An empirical examination. Entrepreneurship and Innovation, 9(4): 231-240.

Hajek, P., & Henriques, R. (2017). Correction: Modelling innovation performance of European regions using multi-output neural networks. PLOS ONE, 12 (2): e0189746.

Hajek, P., Olej, V., & Myskova, R. (2013). Forecasting stock prices using sentiment information in annual reports – A neural network and support vector regression approach. WSEAS Transactions on Business and Economics, 10(4): 293-305.

Harris, R.I.D. (1995). Cointegration Analysis in Econometric Modeling. London: Harvester Wheatsheaf.

Hassouna, I. (2018). Behind the Number: A review of index methodologies to improve innovation measurement in Africa. Working Paper 11 Access to Knowledge for Development Center (A2K4D). American University in Cairo (February 8).

Hebb, D. (2002[1949]. The Organization of Behavior. Online: Psychology Press.

Hecht-Nielsen, R. (1987). Theory of the back propagation neural network. In Proceedings of the International Joint Conference on Neural Networks (pp. 1:593-608). San Diego: SOS Printing.

Herbrich, R., Keilbach, M., Graepel, T., Bollmann-Sdorra, P., & Obermayer, K. (1998). Neural networks in economics: Background, applications, and new developments. Department of Computer Science. Technical University of Berlin, Germany.

Hidalgo, C.A. (2018). The rise of knowledge economics. Scientific American (December 13). https://blogs.scientificamerican.com/observations/the-rise-of-knowledge-economics/. Accessed May 15, 2019.

Hidalgo, C.A., & Hausmann, R. (2009). The building blocks of economic complexity. PNAS, 106(26): 10570-10575.

Hidalgo, C.S, Klinger, B., Barabasi, A.L., & Hausmann, R. (2007). The product space conditions the development of nations. Science 317 (482). Doi: 10.1126/scince.1144581.

Hinton, G. (1987). Learning translation invariant recognition in massively parallel networks. In Proceedings Conference on Parallel Architectures and Languages Europe, pp. 1-13. Springer Verlag.

Hochreiter, S. (1998). The vanishing gradient problem during learning recurrent neural nets and problem solutions. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 6(02): 107-116.

Honaker, J., & King, G. (2010). What to do about missing values in time-series cross-section data?. American Journal of Political Science, 54 (2): 561-581

Hopfield, J.J. (1982). Neural networks and physical systems with emergent collective computational abilities, Proceedings of the National Academy of Sciences of the USA, 79 (8): 2554–2558 (April).

Hopfield, J.J., & Tank, D. (1986). Computing with neural circuits. Science 233: 625- 633.

Hornik, K., Stinchcombe, M., & White, H. (1989). Multilayer feedforward networks are universal approximators. Neural Networks 2(5): 359-366.

Hush, D.R., & Horne, B.G. (1993). Progress in supervised neural networks. IEEE signal processing magazine, 10(1), 8-39.

Jaffe, A.B., Trajtenberg, M., & Henderson, R. (1993). Geographic Localization of Knowledge Spillovers as Evidenced by Patent Citations. The Quarterly Journal of Economics, 108(3): 577-598.

Johnson, R., &Zhang, T. (2013). Accelerating stochastic gradient descent using predictive variance reduction. In Advances in Neural Information Processing Systems (pp. 315-323)

Karsoliya, S. (2012). Approximating number of hidden layer neurons in multiple hidden layer BPNN architecture. International Journal of Engineering Trends and Technology, 3(6): 714-717.

Kennedy, P. (1992). A Guide to Econometrics. Third Edition. Cambridge (MA): The MIT Press. Chapter 14.

Khayyat, N.T., & Lee, J-D. (2012). A new measure of technological capabilities for developing countries. TEMEP Discussion Paper No. 2012:91. Technology Management, Economics, and Policy Program, College of Engineering, Seoul National University.

Kinne, J., & Lenz, D. (2019). Predicting innovative firms using web mining and deep learning, ZEW Discussion Papers, No. 19-001, ZEW - Leibniz-Zentrum für Europäische Wirtschaftsforschung, Mannheim.

Kleinberg, J., Lakkaraju, H.., Leskovec, J., &Mullainathan, S. (2018). Human decisions and machine predictions. The Quarterly Journal of Economics, 133(1): 237–293.

Kuan, C.M., & Liu, T. (1995). Forecasting exchange rates using feedforward and recurrent neural networks. Journal of Applied Econometrics 10: 347-364.

Kuhlman, C., Ramamurthy, K.N., Sattigeri, P., Lozano, A.C., Cao, L., Reddy, C., Mojsilovic, A., & Varshney, K.R. (2017). How to Foster Innovation: A Data-Driven Approach to Measuring Economic Competitiveness. IBM J. Res. Dev. 61(6): 11:1-11:12.

Kuhn, M. (2015). A short introduction to the caret package. R Found Stat Comput: 1-10.

Landes, D. (1998) The Wealth and Poverty of Nations: Why Some Are So Rich and Some So Poor. New York: W.W. Norton

LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature 521(7553): 436.

Lim, H.K., Narrisetty, N.N., & Cheon, S. (2017). Robust multivariate mixture regression models with incomplete data. Journal of Statistical Computation and Simulation, 87(2): 328-347.

Maladkar, K. (2018). 6 types of artificial neural networks currently being used in machine learning. (January 15); https://www.analyticsindiamag.com/6-types-of-artificial-neural-networks-currently-being-used-in-todays-technology/. Accessed May 15, 2019.

McCulloch, W.S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5:115–133.

McDonough, I.K., & Millimet, D.L. (2016). Missing data, imputation, and endogeneity. Journal of Econometrics, 199(2): 141-155.

McNelis, P., & McAdam, P. (2005). Forecasting inflation with thick models and neural networks. Economic Modelling, 22(5): 848-867.

Mihaylova, I. (2018). Applications of Artificial Neural Networks in Economics and Finance. In Encyclopedia of Information Science and Technology, Fourth Edition. Chapter 5755: 6631-6641. https://www.igi-global.com/chapter/applications-of-artificial-neural-networks-in-economics-and-finance/184358.

Miller, J.I. (2009). Cointegrating regressions with messy regressors: Missingness, mixed frequency, and measurement error. Department of Economics, University of Missouri.

Moghaddam, A.H., Moghaddam, M.H., & Esfandyari, M. (2017). Stock market index prediction using artificial neural network. Journal of Economics, Finance and Administrative Science, 21(41): 89-93.

Moshiri, S., & Cameron, N. (2000). Neural network versus econometric models in forecasting inflation. Journal of Forecasting, 19: 201-217.

Mullainathan, S., & Spiess, J. (2017). Machine learning: An applied econometric approach. Journal of Economic Perspectives, 31(2): 87–106.

Nair, H., Kumar, A., & Ahmed, O. (2014). Neural Network Modelling, Simulation and Prediction of Innovation Growth in United Arab Emirates (UAE). Procedia Computer Science, 3: 269-275

Nakamura, E. (2005). Inflation forecasting using a neural network. Economics Letters,86: 373-378.

Neffke, F., Henning, M., & Boschma, R. (2011). How do regions diversify over time? Industry relatedness and the development of new growth paths in regions. Economic Geography, 87 (3): 237-265.

Nobel Prize Committee 2018 Integrating nature and knowledge into economics. The Prize in Economic Sciences2018. https://www.nobelprize.org/uploads/2018/10/popular-economicsciencesprize2018.pdf. Accessed May 15, 2019.

North, D.C. (1990). Institutions, Institutional Change and Economic Performance. Cambridge: Cambridge University Press

Nowak, A., Vallacher, R.R., & Burnstein, E. (1998). Computational social psychology: A neural network approach to interpersonal dynamics. In Computer Modeling of Social process. Edited by Win B.G. Liebrand, Andrzej Nowak and Rainer Hegselman. London: Sage Publications. Chapter 6

Ojanpera, S., Graham, & Zook, M. (2019). The digital knowledge economy index: Mapping content production. Journal of Development Studies. https://doi.org/10.1080/00220388.2018.1554208 (January 30). Accessed May 15, 2019.

Parente, S., & Prescott, E. (1994). Barriers to technology adoption and development. Journal of Political Economy, 102: 298-321.

Parcero, O.J., & Ryan, J.C. (2017). Becoming a Knowledge Economy: The Case of Qatar, UAE, and 17 Benchmark Countries. Journal of Knowledge Economy, 8, 1146–1173.

Pao, H. T., & Chih, Y. Y. (2006). Comparison of TSCS regression and neural network models for panel data forecasting: debt policy. Neural Computing & Applications, 15(2), 117-123.

Rao, B.C. (2010). On the methodology for quantifying innovations. International Journal of Innovation Management, 14 (5): 823-840.

Rizk, N., El Said, A., Weheba, N., and de Beer, J. (2018). Towards an alternative assessment of innovation in Africa. Working Paper 10. Access to Knowledge for Development Center (A2K4D). American University in Cairo. (February 8).

Romer, P.M. (1990). Endogenous technological change. Journal of Political Economy, 98(5): S71-S102.

Romer, P.M. (1994). The origins of endogenous growth. Journal of Economic Perspectives, 8(1): 3-22.

Rukšenaite, J., &bVaitkus, P. (2012). Prediction of composite indicators using combined method of extreme learning machine and locally weighted regression. Nonlinear Analysis: Modelling and Control, 17 (2): 238–251.

Rumelhart, D.E., G. E. Hinton, & R. J. Williams (1986). Learning representations by backpropagating errors. Nature 323: 533-536.

Ryan, G., & Shinnick, E. (2011). Economic incentives and the knowledge economy: In Encyclopedia of Knowledge Management, Second Edition, Chapter 23.

Simpson, P.K. (1990). Artificial Neural Systems. New York: Pergamon Press.

Singh, J. (2004). Collaborative networks as determinants of knowledge diffusion patterns. Harvard Business School. Harvard University. Available at https://flora.insead.edu/fichiersti_wp/inseadwp2004/2004-74.pdf

Soley-Bai, M. (2013). Dealing with missing data: key assumptions and methods for applied analysis. Technical Report No. 4. Boston University, School of Public Health.

Sutz, J. (2012). Measuring innovation in developing countries: Some suggestions to achieve more accurate and useful indicators. International Journal for Technological Learning and Development, 5(1/2): 40-55.

Swanson, N., &White, H. (1997). A model selection approach to real-time macroeconomic forecasting using linear models and artificial neural networks. Review of Economics and Statistics, 79 (4): 540-550.

Tchamyou, S.V. (2016). The role of knowledge economy in African business. Journal of the Knowledge Economy, DOI: 10.1007/s13132-016-0417-

Tibshirani, R., Wainwright, M., & Hastie, T. (2015). Statistical Learning with Sparsity: The Lasso and Generalizations. Chapman and Hall/CRC.

Tkacz, G., & Hu, S. (1999). Forecasting GDP Growth Using Artificial Neural Networks. Working Paper 99-3/ Document de travail 99-3.

Tzafestas, S. G., Dalianis, P. J., &Anthopoulos, G. (1996). On the overtraining phenomenon of backpropagation neural networks. Mathematics and Computers in Simulation, 40(5-6), 507-521.

Vapnik, V. (1982). Estimation of Dependences Based on Empirical Data. New York: Springer-Verlag.

Vapnik, V. (1995). The Nature of Statistical Learning Theory. New York: Springer-Verlag.

Vapnik, V. (1998). Statistical Learning Theory. New York: John Wiley and Sons.

Vapnik, V. & A. Chervonenkis (1971). On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its Application, 16(2): 264-281.

Vapnik, V., & Chervonenkis, A. (1964). A note on one class o perceptrons. Automation and Remote Control, 25.

Vapnik, V., & Lerner, A. (1963). Pattern recognition using generalized portrait method. Automation and Remote Control, 24: 774-780.

Varian, H.R. (2014). Big Data: New Tricks for Econometrics. Journal of Economic Perspectives, 28(2), 3-28

Verkooijen, W. (1996). A neural network approach to long-run exchange rate prediction. Computational Economics, 9: 51-65.

Werbos, P. J. (1990). Backpropagation through time: what it does and how to do it. Proceedings of the IEEE, 78(10): 1550-1560.

White, H. (1988). Economic Prediction Using Neural Networks: The Case of the IBM Daily Stock Returns. Proceedings of IEEE International Conference on Neural Networks, San Diego, 24-27 July 1988, 451-458. http://dx.doi.org/10.1109/ICNN.1988.23959.

World Bank Institute. Measuring knowledge in the world’s economies: Knowledge assessment methodology and knowledge economy index. http://web.worldbank.org/archive/website01030/WEB/IMAGES/KAM_V4.PDF. Accessed May 15, 2019.

Zanak, S.Z., & Becerra-Fernández, I. (2005). Competitiveness of nations: A knowledge discovery examination. European Journal of Operations Research, 166: 185-211

Zaremba, W., Sutskever, I., & Vinyals, O. (2014). Recurrent neural network regularization. arXiv preprint arXiv:1409.2329.

Zeiler, M.D. (2012). ADADELTA: An adaptive learning rate method. arXiv preprint arXiv:1212.5701.

Zhang, S. (2012). Nearest neighbor selection for iteratively kNN imputation. Journal of Systems and Software, 85(11): 2541-2552.

Zhu, G., Zhaoxiang, Zhang, Xu-Yao, & Liu, Cheng-Lin (2017). Diverse neuron type selection for convolutional neural networks. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17): 3560-3566.

Zurada, J.M. (1992). Introduction to Artificial Neural Systems (Vol. 8). St. Paul: West publishing company.