Neural networks as a learning paradigm for general normal form games

Spiliopoulos, Leonidas (2009): Neural networks as a learning paradigm for general normal form games.

Abstract

This paper addresses how neural networks learn to play one-shot normal form games through experience in an environment of randomly generated game payoffs and randomly selected opponents. This agent based computational approach allows the modeling of learning all strategic types of normal form games, irregardless of the number of pure and mixed strategy Nash equilibria that they exhibit. This is a more realistic model of learning than the oft used models in the game theory learning literature which are usually restricted either to repeated games against the same opponent (or games with different payoffs but belonging to the same strategic class). The neural network agents were found to approximate human behavior in experimental one-shot games very well as the Spearman correlation coefficients between their behavior and that of human subjects ranged from 0.49 to 0.8857 across numerous experimental studies. Also, they exhibited the endogenous emergence of heuristics that have been found effective in describing human behavior in one-shot games. The notion of bounded rationality is explored by varying the topologies of the neural networks, which indirectly affects their ability to act as universal approximators of any function. The neural networks' behavior was assessed across various dimensions such as convergence to Nash equilibria, equilibrium selection and adherence to principles of iterated dominance.

• Item Type: MPRA Paper
• Language: English
• Keywords: Behavioral game theory; Learning; Global games; Neural networks; Agent-based computational economics; Simulations; Complex adaptive systems; Artificial intelligence
• Subjects: C - Mathematical and Quantitative Methods > C4 - Econometric and Statistical Methods: Special Topics > C45 - Neural Networks and Related Topics; C - Mathematical and Quantitative Methods > C7 - Game Theory and Bargaining Theory > C70 - General; C - Mathematical and Quantitative Methods > C7 - Game Theory and Bargaining Theory > C73 - Stochastic and Dynamic Games ; Evolutionary Games ; Repeated Games

References

Binmore, K., J. Swierzbinski, and C. Proulx (2001). Does Minimax Work? An Experimental Study. Economic Journal 111(473), 445–464.

Cabrales, A., W. Garcia-Fontes, and M. Motta (2000). Risk dominance selects the leader: An experimental analysis. International Journal of Industrial Organization 18, 137 – 162.

Cho, I. and T. Sargent (1996). Neural Networks for Encoding and Adapting in Dynamic Economies. Handbook of Computational Economics 1, 441–470.

Cooper, R. W., D. DeJong, R. Forsythe, and T. Ross (1994). Problems of Coordination in Economic Activity, Chapter Alternative Institutions for Resolving Coordination Problems: Experimental Evidence on Forward Induction and Preplay Communication. Kluwer Academic Publishers.

Cooper, R. W., D. V. DeJong, R. Forsythe, and T. W. Ross (1990). Selection criteria in coordination games: Some experimental results. The American Economic Review 80(1), 218–233. 53

Costa-Gomes, M., V. P. Crawford, and B. Broseta (2001). Cognition and behavior in normal-form games: An experimental study. Econometrica 69(5), 1193–1236.

Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems 2, 303–314.

Dror, I. E. and D. P. Gallogly (1999). Computational analyses in cognitive neuroscience: In defense of biological implausibility. Psychonomic Bulletin & Review 6(2), 173–182.

Funahashi, K. (1989). On the approximate realization of continuous mappings by neural networks. Neural Networks 2, 183–192.

Germano, F. (2007). Stochastic Evolution of Rules for Playing Finite Normal Form Games. Theory and Decision 62(4), 311–333.

Gilboa, I. and D. Schmeidler (1995). Case-Based Decision Theory. The Quarterly Journal of Economics 110(3), 605–639.

Harsanyi, J. C. and R. Selten (1988). A General Theory of Equilibirum Selection in Games. Cambridge, MA: MIT Press.

Haruvy, E. and D. Stahl (2004). Deductive versus inductive equilibrium selection: experimental results. Journal of Economic Behavior and Organization 53(3), 319–331.

Hensher, D. and L. Johnson (1981). Applied Discrete-choice Modelling. Wiley.

Hornik, K. (1991). Approximation capabilities of multilayer feedforward networks. Neural Networks 4, 251–257.

Hosmer, D. and S. Lemeshow (1989). Applied Logistic Regression. John Wiley & Sons New York.

Huysmans, J., B. Baesens, and J. Vanthienen (2006). Using rule extraction to improve the comprehensibility of predictive models. Technical Report KBI 0612, Katholieke Universiteit Leuven.

Ivanov, A. (2006, October). Strategic play and risk aversion in one-shot normal-form games: An experimental study.

Katz, K. (1996). Three Applications of Game Theory. Ph. D. thesis, University of Pennsylvania.

Kettner, R., J. Marcario, and N. Port (1993). A neural network model of cortical activity during reaching. Journal of Cognitive Neuroscience 5, 14–33.

Kuan, C.-M. and T. Liu (1995). Forecasting exchange rates using feedforward and recurrent neural networks. Journal of Applied Econometrics 10(4), 347–364.

Leake, D. (Ed.) (1996). Case-Based Reasoning: Experiences, Lessons, and Future Directions. AAAI/MIT Press.

LeCun, Y., L. Bottou, G. B. Orr, and K.-R. Muller (1998). Neural Networks: Tricks of the Trade, Chapter Efficient backprop, pp. 9–50. Springer Berlin / Heidelberg.

Lehky, S. R. and T. J. Sejnowski (1988). Network model of shape-from-shading: Neural function arises from both receptive and projective fields. Nature 333, 452–454.

Lemke, C. E. and J. Howson, J. T. (1964). Equilibrium points of bimatrix games. Journal of the Society for Industrial and Applied Mathematics 12(2), 413–423.

Leung, M. T., H. Daouk, and A. S. Chen (2000). Forecasting stock indices: A comparison of classification and level estimation models. International Journal of Forecasting 16, 173–190.

LiCalzi, M. (1995). Fictitious play by cases. Games and Economic Behavior 11, 64–89.

Louviere, J., D. Hensher, and J. Swait (2000). Stated choice methods. Cambridge University Press New York.

Matlab (2007). Mathworks, Inc., Natick, MA.

Mazzoni, P., R. A. Andersen, and M. I. Jordan (1991). A more biologically plausible learning rule than backpropagation applied to a network model of cortical area 7a. Cerebral Cortex 1, 293–307.

McFadden, D., I. of Urban & Regional Development, and B. U. of California (1973). Conditional Logit Analysis of Qualitative Choice Behavior. Institute of Urban and Regional Development, University of California.

Mookherjee, D. and B. Sopher (1997). Learning and Decision Costs in Experimental Constant Sum Games. Games and Economic Behavior 19(1), 97–132.

Myers, R. (1986). Classical and modern regression with applications. Duxbury Press Boston, Mass.

Nagel, R. (1995). Unraveling in Guessing Games: An Experimental Study. The American Economic Review 85(5), 1313–1326.

Nakamura, E. (2005). Inflation forecasting using a neural network. Economics Letters 86(373-378).

Nguyen, D. and B. Widrow (1990). Improving the learning speed of 2–layer neural network by choosing initial values of the adaptive weights. In IEEE First International Joint Conference on Neural Networks, pp. 21–26.

Ockenfels, A. and R. Selten (2005). Impulse Balance Equilibrium and Feedback in First Price Auctions. Games and Economic Behavior 51, 155–170.

Pao, Y. (1989). Adaptive pattern recognition and neural networks. Addison-Wesley Longman Publishing Co., Inc. Boston, MA, USA.

Rey Biel, P. (2004). Equilibrium Play and Best Response to (Stated) Beliefs in Constant Sum Games. Technical report, mimeo.

Rieskamp, J., J. Busemeyer, and T. Laine (2003). How do people learn to allocate resources? Comparing two learning theories. Journal of experimental psychology. Learning, memory, and cognition 29(6),1066–1081.

Robinson, T. (2000). Biologically plausible back-propagation. Technical report, Victoria University of Wellington.

Sargent, T. S. (1993). Bounded Rationality in Macroeconomics. Clarendon Press.

Sarkar, D. (1995). Methods to speed up error back-propagation learning algorithm. ACM Computing Surveys 27(4), 519–544.

Sarle, W. (1994). Neural networks and statistical models. Proceedings of the Nineteenth Annual SAS Users Group International Conference, 1538–1550.

Schotter, A., K. Weigelt, and C. Wilson (1994). A laboratory investigation of multiperson rationality and presentation effects. Games and Economic Behavior 6(3), 445–468.

Selten, R. (1998). Features of Experimentally Observed Bounded Rationality. European Economic Review 42(3-5), 413–36.

Selten, R., K. Abbink, and R. Cox (2005). Learning Direction Theory and the Winner’s Curse. Experimental Economics 8(1), 5–20.

Sgroi, D. and D. J. Zizzo (2002). Strategy Learning in 3x3 Games by Neural Networks. Technical report, Department of Applied Economics, University of Cambridge.

Sgroi, D. and D. J. Zizzo (2007). Neural networks and bounded rationality. Physica A: Statistical Mechanics and its Applications 375(2), 717–725.

Smith, K. A. and J. N. Gupta (2002). Neural networks in business : techniques and applications. Idea Group Publishing.

Stahl, D. and E. Haruvy (2004). Rule Learning Across Dissimilar Symmetric Normal-Form Games.

Stahl, D. O. and P. W. Wilson (1995). On Players’ Models of Other Players: Theory and Experimental Evidence. Games and Economic Behavior 10(1), 218–254.

Steiner, J. and C. Stewart (2006). Learning by similarity in global games. http://www.econ.ed.ac.uk/papers/Learning_by_Similarity.pdf.

Straub, P. G. (1995). Risk dominance and coordination failures in static games. The Quarterly Review of Economics and Finance 35(4), 339–363.

Tang, F. F. (2001). Anticipatory learning in two-person games: some experimental results. Journal of Economic Behavior & Organization 44(2), 221–232.

Tesfatsion, L. (2002). Agent-based computational economics: Growing economies from the bottom up. Artificial Life 8(1), 55–82.

Tesfatsion, L. and K. L. Judd (2006). Handbook of Computational Economics Volume 2. Elsevier/North-Holland (Handbooks in Economics Series).

Walczak, S. and N. Cerpa (1999). Heuristic principles for the design of artificial neural networks. Information and Software Technology 41, 107–117.

Waldrop, M. (1992). Complexity: The emerging science at the edge of order and chaos. Simon and Schuster: New York.

Yang, Z. R., M. B. Platt, and H. D. Platt (1999). Probabilistic neural networks in bankruptcy prediction. Journal of Business Research 44, 67–74.

Zipser, D. and R. A. Andersen (1988). A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons. Nature 331, 679–684.