Network Formation with Adaptive Agents

Schuster, Stephan (2010): Network Formation with Adaptive Agents.

Abstract

In this paper, a reinforcement learning version of the connections game first analysed by Jackson and Wolinsky is presented and compared with benchmark results of fully informed and rational players. Using an agent-based simulation approach, the main nding is that the pattern of reinforcement learning process is similar, but does not fully converge to the benchmark results. Before these optimal results can be discovered in a learning process, agents often get locked in a state of random switching or early lock-in.

• Item Type: MPRA Paper
• Language: English
• Keywords: agent-based computational economics; strategic network formation; network games; reinforcement learning
• Subjects: C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling > C63 - Computational Techniques ; Simulation Modeling; D - Microeconomics > D8 - Information, Knowledge, and Uncertainty > D85 - Network Formation and Analysis: Theory

References

W. Brian Arthur. On designing economic agents that behave like human agents. Journal of Evolutionary Economics, 3(1):1–22, 1993.

V. Bala and S. Goyal. A noncooperative model of network formation. Econometrica, 68(5):1181–1229, 2000.

Sylvain Beal and Nicolas Querou. Bounded rationality and repeated network formation. Mathematical Social Sciences, 54:71–89, 2007.

A. W. Beggs. On the convergence of reinforcement learning. Journal of Economic Theory, 122:1–36, 2005.

T. Boergers and R. Sarin. Learning through reinforcement and replicator dynamics. Journal of Economic Theory, 77:1–14, 1997.

Tilman Boergers and Rajiv Sarin. Naive learning with endogenous aspirations. International Economic Review, 41(4):921–950, 2000.

Colin F. Camerer, Juin-Kuan Chong, and Teck H. Ho. Self-tuning experience weighted attraction learning in games. Journal of Economic Theory, 133:177–198, 2007.

Colin F. Camerer and Teck H. Ho. Experience-weighted attraction learning in normal form games. Econometrica, 67(4):827–874, 1999.

Yan Chen and Yuri Khoroshilov. Learning under limited information. Games and Economic Behavior, 44:1–25, 2003.

Yan Chen and Fang-Fang Tang. Learning and incentive-compatible mechanisms for public goods provision: An experimental study. Journal of Political Economy, 106(3):633–662, 1998.

Frederic Deroian. Farsighted strategies in the formation of a communication network. Economic Letters, 80:343–349, 2003.

Patrick Doreian. Actor network utilities and network evolution. Social Networks, 28(2):137–164, 2006.

B. Dutta and S. Mutuswami. Stable networks. Journal of Economic Theory, 76:251–272, 1997.

I. Erev and A. Roth. Predicting how people play games: reinforcement learning in experimental games with unique mixed-strategy equilibria. American Economic Review, 88, 1998.

Andrea Galeotti, Sanjeev Goyal, and Jurjen Kamphorst. Network formation with heterogeneous players. Games and Economic Behavior, 54:353–372, 2006.

Jacob K. Goeree, Arno Riedl, and Aljaz Ule. In search of stars: Network formation among heterogeneous agents. Games and Economic Behavior, 67:445–466, 2009.

Nicholas M. Gotts, Luis R. Izquierdo, Segismundo S. Izquierdo, and J. Gary Polhill. Transient and asymptotic dynamics of reinforcement learning in games. Games and Economic Behavior, 61:259–276, 2007.

Sanjeev Goyal. Learning in networks: a survey. University of Essex, 2003.

Sanjeev Goyal. Connections. Princeton University Press, 2007.

Ed Hopkins and Martin Posch. Attainability of boundary points under reinforcement learning. Games and Economic Behavior, 53:110–125, 2005.

N.P. Hummon. Utility and dynamic social networks. Social Networks, 22:221–249, 2000.

Matthew O. Jackson. Social and Economic Networks. Princeton University Press, 2008.

Matthew O. Jackson and Anne van den Nouweland. Strongly stable networks. Games and Economic Behavior, 51:420–444, 2005.

M.O. Jackson and A.Watts. The evolution of social and economic networks. Journal of Economic Theory, 106:265–295, 2002.

M.O. Jackson and A. Wolinsky. A strategic model of social and economic networks. Journal of Economic Theory, 71:44–74, 1996.

Rajeeva Karandikar, Dilip Mookherjee, Debraj Ray, and Fernando Vega-Redondo. Evolving aspirations and cooperation. Journal of Economic Theory, 80:292–331, 1998.

Jean-Franois Laslier, Richard Topol, and Bernard Walliser. A behavioral learning process in games. Games and Economic Behavior, 37:340–366, 2001.

Robert D. Luce. Individual Choice Behavior: A Theoretical Analysis. Wiley, New York, 1959.

Michael McBride. Imperfect monitoring in communication networks. Journal of Economic Theory, 126:97–119, 2006.

D. Mookherjee and B. Sopher. Learning and decision costs in experimental constant-sum games. Games and Economic Behaviour, 19:97–132, 1997.

R. Pemantle and B. Skyrms. A dynamic model of network formation. Proceedings of the National Academies of Science, 97:9340–9346, 2000.

R. Pemantle and B. Skyrms. Network formation by reinforcement learning: the long and medium run. Mathematical Social Sciences, 48(3):315–327, November 2004.

A. Roth and I. Erev. Learning in extensive form games: Experimental data and simple dynamic models in the intermediate run. Games and Economic Behaviour, 6, 1995.

Rajiv Sarin and Farshid Vahid. Payoff assessments without probabilities: A simple dynamic model of choice. Games and Economic Behavior, 28:294–309, 1999.

Stephan Schuster. Bra: An algorithm for simulating bounded rational agents. Computational Economics, pages 1–19, 2010. 10.1007/s10614-010-9231-1.

Marco Slikker and Ann van den Nouweland. Network formation models with costs for establishing links. Review of Economic Design, 5:333–362,2000.

A. Watts. A dynamic model of network formation. Games and Economic Behaviour, 34:331–341, 2001.

Alice Watts. Non-myopic formation of circle networks. Economic Letters,74:277–282, 2002.