The Neuroeconomics of Learning and Information Processing; Applying Markov Decision Process

Chatterjee, Sidharta (2011): The Neuroeconomics of Learning and Information Processing; Applying Markov Decision Process.

Abstract

This paper deals with cognitive theories behind agent-based modeling of learning and information processing methodologies. Herein, I undertake a descriptive analysis of how human agents learn to select action and maximize their value function under reinforcement learning model. In doing so, I have considered the spatio-temporal environment under bounded rationality using Markov Decision process modeling to generalize patterns of agent behavior by analyzing the determinants of value functions, and of factors that modify policy- action-induced cognitive abilities. Since detecting patterns are central to the human cognitive skills, this paper aspires at uncovering the entanglements of complex contextual pattern identification by linking contexts with optimal decisions that agents undertake under hypercompetitive market pressure through learning which have however, implicative applications in a wide array of social and macroeconomic domains.

• Item Type: MPRA Paper
• Language: English
• Keywords: Cognitive theory, Reinforcement Learning, Markov Decision Process, Glia, Action potential, policy pattern, Neuroeconomics
• Subjects: D - Microeconomics > D8 - Information, Knowledge, and Uncertainty > D81 - Criteria for Decision-Making under Risk and Uncertainty; C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling > C61 - Optimization Techniques ; Programming Models ; Dynamic Analysis; D - Microeconomics > D8 - Information, Knowledge, and Uncertainty > D87 - Neuroeconomics

References

Alchian, A. (1950) ‘Uncertainty, Evolution and Economic Theory’, Journal of Political Economy, vol. 58: 211-221.

Allais, M. (1952), “Le comportament de l'homme rationnel devant le risque: Critique des postulats de l'école Américaine”, Econometrica, 21, pp. 503-46.

Arthur, W. Brian. (2005). Paper prepared for Handbook of Computational Economics, Vol. 2: Agent-Based Computational Economics, K. Judd and L. Tesfatsion, eds, ELSEVIER/North-Holland.

Arthur, W. Brian., Durlauf, Steven, and Lane, David. (1997). The Economy as an Evolving Complex System II. Introduction to the volume.

Atkinson, J.W. (1957). ‘Motivational determinants of risk-taking behavior’. Psychological Review, 64; 359-372.

Bayer, C- Ralph, and Renou, Ludovic. (2011). Cognitive abilities and behavior in strategic-form games.University of Leicester, Department of economics, Working Paper no. 11/16.

Beauchamp M.H., Dagher A., Panisset, M. and Doyon, J. (2008). Behavioural Correlates of Cognitive Skill Learning in Parkinson’s Disease, The Open Behavioral Science Journal, 2008, 2, 1-12.

Bem, D.J., and Allen, A.(1974) On predicting some of the people some of the time. Psychological Review, 81, 506-520.

Bently, David., Konishi, Masakazu.(1978). Neural control of behavior. Ann. Rev. Neurosci. 1:35(59).

Bracha, Anat, Brown J. Donald; (2010) Affective Decision Making: A Theory of Optimism Bias, Fed. Res. Bank Boston Working Paper no. 10-16.

Clint Ballinger. 2008. Classifying Contingency in the Social Sciences: Diachronic, Synchronic, and Deterministic Contingency, Cambridge University Paper.

D.J. McFarland. (1970). Behavioral aspects of homeostasis’. Advances in the study of behavior, vol. 3, 1-26.

David P. (1997) Path - Dependence and the Quest for Historical Economics: One More Chorus of the Ballad of QWERTY. Discussion Paper in Economic and Social History. Oxford: University of Oxford.

Dosi, G., Faillo, Marco., Marengo, Luigi., Moschella, D. (2011). Modeling routines and organizational learning. A discussion of the state-of-art. Laboratory of economics and management, Sant’ Anna School of Advanced Studies, Italy, Working Paper Series, 2011/04.

Egidi M (2002), "Biases in Organizational Behavior", in M Augier and J.J. March (eds), The Economics of Choice, Change and Organization: Essays in Memory of Richard M. Cyert, Aldershot, Edward Elgar.

Ferguson, Niall. 1997. Virtual history: Towards a ‘chaotic’ theory of the past. In Virtual history: Alternatives and counterfactuals, Niall Ferguson ed. London: Picador.

Fields, R.D., Stevens-Graham, B. 2002. New Insights into neuron-glia communication, Sience: Vol. 298, 556-562.

Forman, S. Mark, Trojanowski, Q. John, Lee, M-Y Virginia, 2004. Neurodegenerative Diseases: A decade of discoveries paves the way for therapeutic break-through. Nature Medicine, Vol. 10, 1055-1063.

Friedman M. (1953), Essays in Positive Economics, The University Press, Chicago.

Hall, A.D., and R.E. Fagen (1956), “Definitions of a System”, in L. von Bertalanffy and A. Rapoport (eds.), General Systems: Volume I. Ann Arbor: University of Michigan Press.

Hayek, F.A. (1952). The Sensory Order. An Inquiry into the Foundations of Theoretical Psychology, London: Routledge & Kegan Paul.

Heider, F. (1944). Social perception and phenomenon causality. Psychological Review, 51: 358-373.

Holland J.H., K. Holyoak. (1986), R. Nisbett, P. Thagard, Induction, MIT Press.

Houser, Daniel, Keane, Michael, and McCabe Kevin (2004); Behavior in a dynamic decision problem: An analysis of experimental evidence using a bayesian type classification algorithm. Econometrica: Vol. 72, 781-822.

Kahneman, Daniel and Tversky, Amos. (1979). Prospect theory: An analysis of decision under risk. Econometrica, Vol.47, No. 2, 263-292.

Kuljiš O. Rodrigo:(2010) Integrative Understanding of Emergent Brain Properties, Quantum Brain Hypotheses, and Connectome Alterations in Dementia are Key Challenges to Conquer Alzheimer's Disease. Front Neurol. 2010; 1: 15.

Lang A.E., Lozano, A.M. 1998. Parkinson’s disease, Parts 1and 2. New England Journal of Medicine, Vol.339, 1044-1053 and 1130-1143.

Mahoney, James. 2000. Path dependence in historical sociology. Theory and Society 29:507-548

Marengo, L., G. Dosi, P. Legrenzi and C. Pasquali (2000), “The Structure of Problem-Solving Knowledge and the Structure of Organizations”, Industrial and Corporate Change, 9, 757-788.

Marshall, A. (1867-8), Ye Machine, in Raffaelli (1994).

Marshall, A. (1890), Principles of Economics, London: MacMillan.

Mongin, Philippe; Expected Utility Theory Handbook of Economic Methodology (J. Davis, W. Hands, and U. Maki, eds. London, Edward Elgar, 1997, p. 342-350).

Novarese, Marco and Lanteri, Alessandro. 26 April 2007. Individual learning: theory formation, and feedback in a complex task Centre for Cognitive Economics, Universit`a Amedeo Avogadro: MPRA Paper No.3049.

Novarese, Marco, Rizzello, Salvatore. (2004) The Intermingling Between Cognitive Economics and Experimental Economics: a Few Remarks on History, Methodology and Applications. . Dipartimento di Economia “S. Cognetti de Martiis, Centro di Studi sulla Storia e i Metodi dell’Economia Politica "Claudio Napoleoni" (CESMEP)Working paper No. 06.

Novarese, Marco. Is bounded rationality a capacity, enabling learning? EERI Research Paper Series No 12/2009

Oefner, Marc:(2008) Agent based Keynesian Macroeconomics-An evolutionary model embedded in an agent-based computer simulation. Inaugural Dissertation.

Rizzello, Salvatore, Egidi, Massimo. (2003). Cognitive Economics: Foundations and Historical Evolution. Dipartimento di Economia “S. Cognetti de Martiis, Centro di Studi sulla Storia e i Metodi dell’Economia Politica "Claudio Napoleoni" (CESMEP) Working paper No. 04.

Rizzello, Salvatore. (2003). Towards a cognitive evolutionary economics. Dipartimento di Economia “S. Cognetti de Martiis, Centro di Studi sulla Storia e i Metodi dell’Economia Politica "Claudio Napoleoni" (CESMEP) Working paper No. 03.

Rizzello, Salvatore. 2004. Knowledge as Path-dependence Process, Journal of Bioeconomics, 6, 255 – 274.

Roget, P.M. (1834)-“Animal and vegetable psychology considered with refernce to natural theology,” 2 vols., Pickering, London.

Simon A. Herbert. (1978). Rational decision-making in business organizations. Nobel Memorial Lecture, 8 December, 1978. Carnegie-Mellon University, Pittsburgh, Pennsylvania, USA.

Simon A. Herbert; (2000), Bounded Rationality in Social Science: Today and Tomorrow, Mind & Society, Vol 1, n. 1, 25-40.

Simon, A. Herbert (1976). “From Substantive to Procedural Rationality”, in: S. Latsis (ed.) Method and Appraisal in Economics. Cambridge: Cambridge University Press. pp.129-148.

Simon, A. Herbert. (1999). The many shapes of knowledge. Revue d'économie industrielle. Vol. 88. 2e trimestre 1999. pp. 23-39.

Stevens, B., Porta S., Haak, L.L., Gallo V., Fields, R.D. 2002. Adenosine: A neuron-glial transmitter promoting myelination in the CNS in response to action potential. Neuron, Vol. 36, No.5, 855-868.

Sul Jai-Yoon, Orosz George, Givens S. Richard, Haydon G. Philip. 2004. Astrocyte connectivity in the hippocampus. Neuron Glia Biology, Vol.1, 3-11.

Thorndike, Edward Lee (1911), Animal Intelligence: Experimental Studies, New York, Macmillan.