Applications in Agent-Based Computational Economics

Schuster, Stephan (2012): Applications in Agent-Based Computational Economics.

Abstract

A constituent feature of adaptive complex system are non-linear feedback mechanisms between actors. This makes it often difficult to model and analyse them. Agent-based Computational Economics (ACE) uses computer simulation methods to represent such systems and analyse non-linear processes.

The aim of this thesis is to explore ways of modelling adaptive agents in ACE models. Its major contribution is of a methodological nature. Artificial intelligence and machine learning methods are used to represent agents and learning processes in ACE models.

In this work, a general reinforcement learning framework is developed and realised in a simulation system. This system is used to implement three models of increasing complexity in two different economic domains. One of these domains are iterative games in which agents meet repeatedly and interact. In an experimental labour market, it is shown how statistical discrimination can be generated simply by means of the learning algorithm used. The aim of this model is mainly to illustrate the features of the learning framework. The results resemble actual patterns of observed human behaviour in laboratory settings. The second model treats strategic network formation. The main contribution here is to show how agent-based modelling helps to analyse non-linearity that is introduced when assumptions of perfect information and full rationality are relaxed. The other domain has a Health Economics background. The aim here is to provide insights of how the approach might be useful in real-world applications. For this, a general model of primary care is developed, and the implications of different consumer behaviour (based on the learning features introduced before) analysed.

• Item Type: MPRA Paper
• Language: English
• Keywords: agent-based economics, reinforcement learning, statistical discrimination, health care market, network formation games
• Subjects: C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling > C63 - Computational Techniques ; Simulation Modeling; D - Microeconomics > D8 - Information, Knowledge, and Uncertainty > D85 - Network Formation and Analysis: Theory; I - Health, Education, and Welfare > I1 - Health > I11 - Analysis of Health Care Markets; J - Labor and Demographic Economics > J7 - Labor Discrimination > J71 - Discrimination; Y - Miscellaneous Categories > Y4 - Dissertations (unclassified) > Y40 - Dissertations (unclassified)

References

Anderson J (1993) Rules of the mind. Lawrence Erlbaum Associates Inc., Hillsdale

Appleby J, Dixon J (2004) Patient choice in the NHS - Having choice may not improve health outcomes. British Medical Journal 329(7457):61–62

Arrow (1963) Uncertainty and the Welfare Economics of Medical Care. The American Economic Review 53(5):941–973 Arrow K (1973) The theory of discrimination. In: Ashenfelter O, Rees A (eds) Discrimination in Labor markets, Princeton University Press, Princeton, NJ

Arthur WB (1993) On designing economic agents that behave like human agents. Journal of Evolutionary Economics 3(1):1–22

Ascape (2010) The Ascape project, URL http://www.brook.edu/es/dynamics/models/ascape/, last accessed June 2010

Auman R (1997) Rationality and Bounded Rationality. Games and Economic Behaviour 21:2–14

Bala V, Goyal S (2000) A Noncooperative Model of Network Formation. Econometrica 68(5):1181–1229 200

Beal S, Querou N (2007) Bounded rationality and repeated network formation. Mathematical Social Sciences 54:71–89 Beggs AW (2005) On the convergence of reinforcement learning. Journal of Economic Theory 122:1–36

Bendor J, Mookherjee D, Ray D (2001a) Aspiration-based Reinforcement Learning in Repeated Interaction Games: An Overview. International Game Theory Review 3(2-3):159–174

Bendor J, Mookherjee D, Ray D (2001b) Reinforcement Learning in Repeated Interaction Games. Advances in Theoretical Economics 1(1):1–42

Boergers T, Sarin R (1997) Learning Through Reinforcement and Replicator Dynamics. Journal of Economic Theory 77:1–14

Boergers T, Sarin R (2000) Naive learning with endogenous aspirations. International Economic Review 41(4):921–950

Brenner T (2006) Agent learning representation: Advice on modelling economic learning. In: Tesfatsion L, Judd K (eds) Handbook of Computational Economics, 2, Elsevier, chap 18, pp 896–942

Bush R, Mosteller F (1955) Stochastic Models for Learning. Wiley, New York

Butz M (2002) An algorithmic description of ACS2. In: Lanzi P, Stolzmann W, Wilson S (eds) Advances in learning classifier systems, Lecture Notes in Artificial Intelligence, Springer, Berlin, vol 2321, pp 211–229

Camerer CF, Ho TH (1999) Experience-weighted attraction learning in normal form games. Econometrica 67(4):827–874

Camerer CF, Chong JK, Ho TH (2007) Self-tuning experience weighted attraction learning in games. Journal of Economic Theory 133:177–198

Cecconi F, Parisi D (1998) Inividual versus social survival strategies. Jornal of Artificial Societes and Social Simulation 1(2)

Chalkley M, Malcomson J (1998a) Contracting for health services when patient demand does not reflect quality. Journal of Health Economics 17:1–19

Chalkley M, Malcomson J (1998b) Contracting for health services with unmonitored quality. Economic Journal 108(449):1093–1110

Chen Y, Khoroshilov Y (2003) Learning under limited information. Games and Economic Behavior 44:1–25

Chen Y, Tang FF (1998) Learning and Incentive-Compatible Mechanisms for Public Goods Provision: An Experimental Study. Journal of Political Economy 106(3):633–662

Cicirelli F, Furfaro A, Giordano A, Nigro L (2009) Distributed Simulation of RePast Models over HLA/Actors. In: Turner SJ, Roberts D, Cai W, El-Saddik A (eds) DS-RT, IEEE Computer Society, pp 184–191 Cioffi-Revilla C, Luke S, Panait L, Sullivan K (2004)

MASON: A New Multi- Agent Simulation Toolkit. In: Proceedings of the 2004 SwarmFest Workshop Corbae D, Duffy J (2008) Experiments with network formation. Games and Economic Behavior 64:81–120

Davidson S, Manheim L, Werner S, Hohlen M, Yudkowsky B, Fleming G (1992) Prepayment with office-based physicians in publicly funded programs: results from the Childrens Medicaid Program. Pediatrics 89(4):761–767

Department of Health (2006) Our Health, Our Care, Our Say: A new direction for community services, URL www.dh.gov.uk/prod_consum_dh/ idcplg?IdcService=GET_FILE&dID=456&Rendition=Web, last accessed June 2010

Department of Health (2010) GP patient surveys, URL http://www. dh.gov.uk/en/Publicationsandstatistics/PublishedSurvey/ GPpatientsurvey2007/index.htm, last accessed June 2010

Deroian F (2003) Farsighted strategies in the formation of a communication network. Economic Letters 80:343–349

Devlin R, Sisira S (2008) Do physician remuneration schemes matter? The case of Canadian family physicians. Journal of Health Economics 27:1168– 1181

Dixon A, Robertson R (2008) Patient choice in general practice: the implications of patient satisfaction surveys. Journal of Health Services Research & Policy 13(2):67–72

Doreian P (2006) Actor network utilities and network evolution. Social Networks 28(2):137–164

Dranove D (1985) Demand Inducement of the Physician-Patient Relationship. Working Paper, University of Chicago Dutta B, Mutuswami S (1997) Stable Networks. Journal of Economic Theory 76:251–272

Edmonds B, Moss S (2005) From KISS to KIDS? an ’anti-simplistic’ modelling approach. In: Davidson P (ed) Lecture Notes in Artificial Intelligence: Multi Agent Based Simulation 2004, Springer, vol 3415, pp 130– 144

Epstein JM, Axtell R (1996) Growing Artificial Societies. Brookings Institution Press and MIT Press

Erev I, Roth A (1998) Predicting how people play games: reinforcement learning in experimental games with unique mixed-strategy equilibria. American Economic Review 88

Evans R (1976) Modelling the Economic Objectives of the Physician. In: Fraser R (ed) Health Economics Symposium, Queen’s University and Kingston, Canada, Industrial Relations Centre, Queen’s University. Kingston, Ont.

Fagiolo G, Moneta A, Windrum P (2007) A Critical Guide to Empirical Validation of Agent-Based Models in Economics: Methodologies, Procedures, and Open Problems. Computational Economics 30:195–226

Falk A, Kosfeld M (2003) It’s all about Connections: Evidence on Network Formation, Institute for the Study of Labor, Discussion Paper No. 777

Flache A, Macy MW (2002) Learning dynamics in social dilemmas. Proceedings of the National Academy of Science 99:7229–72,346

Fryer Jr RG, Goeree JK, Holt CA (2005) Experience-Based Discrimination: Classroom Games. Journal of Economic Education 36(2):160–170 Gabbott M, Hogg G (1993) Choice of GP: Fact or Fiction. Journal of Management in Medicine 7(1):57–63

Gage H, Rickman N (2000) Patient Choice and Primary Care, Department of Economics, University of Surrey

Galeotti A, Goyal S, Kamphorst J (2006) Network formation with heterogeneous players. Games and Economic Behavior 54:353–372

Gaynor M (2006) What Do We Know About Competition and Quality in Health Care Markets?, Carnegie Mellon University, University of Bristol

Gifford S (2005) Limited Attention as the Bound on Rationality. Contributions to Theoretical Economics 5(1)

Gilbert N (2006) When Does Social Simulation Need Cognitive Models? In: Sun R (ed) Cognition and Multi-Agent Interaction: From Cognitive Modeling to Social Simulation, Cambridge University Press, chap 19, pp 428–433

Gilboa I, Schmeidler D (1996) Case-Based Optimization. Games and Economic Behavior 15:1–26

Goeree JK, Riedl A, Ule A (2009) In search of stars: Network formation among heterogeneous agents. Games and Economic Behavior 67:445–466

Goodwin G (1998) GP Fundholding. In: Le Grand J, Mays N, Mulligan J (eds) Learning from the NHS internal market: A review of the evidence, King’s Fund, London

Gosden T, F F, Kristiansen I, Sutton M, Leese B, Giuffrida A, Sergison M, Pedersen L (2000) Capitation, salary, fee-for-service and mixed systems of payment: effects on the behaviour of primary care physicians (Review). Cochrane Database of Systematic Reviews 3

Gotts NM, Izquierdo LR, Izquierdo SS, Polhill JG (2007) Transient and asymptotic dynamics of reinforcement learning in games. Games and Economic Behavior 61:259–276

Goyal S (2003) Learning in networks: a survey, URL http://www.essex. ac.uk/economics/discussion-papers/papers-text/dp563.pdf, University of Essex

Goyal S (2007) Connections. Princeton University Press

Gravelle H, Masiero G (2000) Quality incentives in a regulated market with imperfect information and switching costs: capitation in general practice. Journal of Health Economics 19:1067–1088

Gravelle H, Sutton M, Ma A (2007) Doctor behaviour under a Pay forPerformance Contract: Evidence from the Quality and Outcomes Framework. Working Paper 28, Centre for Health Economics, University of York

Gigerenzer G, Goldstein DG (1996) Reasoning the fast and Frugal Way: Models of Bounded Rationality. Psychological Review 103(4):650–699

Grignon M, V P, Polton D (2002) Influence of Physician Payment Methods on the Efficiency of the Health Care System. Discussion Paper No 35, Commission on the Future of Health Care in Canada

Grytten J, Srensen R (2001) Type of contract and supplier-induced demand for primary physicians in Norway. Journal of Health Economics

Grytten J, Srensen R (2007) Primary physician servicesList size and primary physicians service production. Journal of Health Economics 26:721–741

Hickson G, Altemeier W, JM P (1987) Physician reimbursement by salary or fee-for-service: effect on physician practice behavior in a randomized prospective study. Pediatrics 80(3):344–350

Hole A (2008) Modelling heterogeneity in patients’ preferences for the attributes of a general practitioner appointment. Journal of Health Economics 27:1078–1094

Holland J (1975) Adaptation in natural and artificial systems: An introductory analysis with application to biology, control, artificial intelligence. University of Michigan Press, Ann Arbor

Hopkins E, Posch M (2005) Attainability of boundary points under reinforcement learning. Games and Economic Behavior 53:110–125

Hummon N (2000) Utility and dynamic social networks. Social Networks 22:221–249

Hutchinson B, Birch S, Hurley J, Lomas J, Stratford-Devai F (1996) Do physician-payment mechanisms affect hospital utilisation? A study of Health Service Organisations in Ontario. Canadian Medical Association Journal 154:653–661

Irwin J, Kiczales G, Lamping J, Loingtier Jm, Lopes C, Maeda C, Mendhekar A (1997) Aspect-Oriented Programming. In: Proceedings of the European Conference on Object-Oriented Programming, vol 1241, pp 220–242

Jackson M, Watts A (2002) The Evolution of Social and Economic Networks. Journal of Economic Theory 106:265–295

Jackson M, Wolinsky A (1996) A Strategic Model of Social and Economic Networks. Journal of Economic Theory 71:44–74

Jackson MO (2008) Social and Economic Networks. Princeton University Press

Jackson MO, van den Nouweland A (2005) Strongly stable networks. Games and Economic Behavior 51:420–444

Jelovac I (2001) Physicians payment contracts, treatment decisions and diagnosis accuracy. Health Economics 10(1):9–25

Jelovac I, Marinoso BG (2003) GPs’ payment contracts and their referral practice. Journal of Health Economics 22:617–635

JGroups (2010) The JGroups project, URL http://www.jgroups.org/, last accessed June 2010

Jones L, Mays N (2010) Systematic review of the impact of patient choice of provider in the English NHS, Health Services Research Unit Department of Public Health and Policy

Kahnemann D, Tversky A (1979) Prospect Theory: An Analysis of Decision under Risk. Econometrica 27:263292

Karandikar R, Mookherjee D, Ray D, Vega-Redondo F (1998) Evolving Aspirations and Cooperation. Journal of Economic Theory 80:292–331

Karlsson M (2007) Quality incentives for GPs in a regulated market. Journal of Health Economics 26:699–720

Kernick D (2006) Wanted: new methodologies for health service research. Is complexity theory the answer? Journal of Health Economics 23:285–390

Kirman A, Vriend N (2001a) Evolving market structure: An ACE model of price dispersion and loyalty. Journal of Economic Dynamics & Control 25:459–502

Kirman A, Vriend NJ (2001b) Evolving market structure: An ACE model of price dispersion and loyalty. Journal of Economic Dynamics and Control 25:459–502

Krasnik FH, Groenewegen P (1992) Introducing fees for services with professional uncertainty. Health Care Financing Review 14(1):107–115

Kristiansen I, Hjortdahl P (1992) The general practitioner and laboratory utilization: why does it vary? Family Practice 9:22–27

Kristiansen I, Holtedahl K (1993) The effect of the remuneration system on the general practitioners choice between surgery consultations and home visits. Journal of Epidemiology and Community Health 47:481–484

Kristiansen I, Mooney G (1993) The general practitioners use of time: is it influenced by the remuneration system? Social Science and Medicine 37:393–399

Laslier JF, Walliser B (2005) A reinforcement learning process in extensive form games. International Journal of Game Theory 33:219–227

Laslier JF, Topol R, Walliser B (2001) A Behavioral Learning Process in Games. Games and Economic Behavior 37:340–366

LeBaron B, Arthur W, Palmer R (1999) Time series properties of an artificial stock market. Journal of Economic Dynamics & Control 23:1487–1516

Lehman J, Laird J, Rosenbloom P (2003) A Gentle Introduction to Soar, an Architecture for Human Cognition, URL http://ai.eecs.umich.edu/ soar/sitemaker/docs/misc/Gentle.pdf, University of Michigan

Leombruni R, Richiardi M (2005) Why are economists sceptical about agent-based simulations? Physica A 355:103–109

Lerner C, Claxton K (1996) Modelling the Behaviour of General Practitioners. Discussion paper 116, Centre for Health Economics, University of York

Levaggi R, Rochaix L (2007) Exit, choice or loyality: Patient driven competition in primary care. Annals of Public and Cooperative Economics 78(4):501–535

Lopes LL (1994) Psychology and Economics: Perspectives on Risk, Cooperation and the Marketplace. Annual Review of Psychology 45:197–227

Luce RD (1959) Individual Choice Behavior: A Theoretical Analysis. Wiley, New York

Ma C (1994) Health care payment systems: cost and quality incentives. Journal of Economics Management and Strategy 3(93-112)

Marinoso B, Jelovac I (2003) GPs’ payment contracts and their referral practice. Journal of Health Economics 22(4):617–635

Markham A (1999) Knowledge Representation. Lawrence Erlbaum Associates, Mawah NJ

MASON (2010) The MASON project, URL http://cs.gmu.edu/~eclab/ projects/mason, last accessed June 2007

McBride M (2006) Imperfect monitoring in communication networks. Journal of Economic Theory 126:97–119

McGuire A, Rickman N (1999) Regulating provider’s reimbursement in a mixed market for healthcare. Scottish Journal of Political Economy 46(1):53–71

Minsky M (1975) A framework for the representation of knowledge. In: Winston P (ed) The pschology of computer vision, McGraw Hill, New York, pp 211–277

Minson R, Theodoropoulos GK (2004) Distributing RePast Agent-Based Simulations with HLA. Concurrency and Computation: Practice and Experience 00:1–25

Mookherjee D, Sopher B (1994) Learning Behaviour in an Experimental Matching Pennies Game. Games and Economic Behaviour 7:62–91

Mookherjee D, Sopher B (1997) Learning and Decision Costs in Experimental Constant-sum Games. Games and Economic Behaviour 19:97–132

Mooney G, Ryan M (1993) Agency in health care: getting beyond first principles. Journal of Health Economics 12(2):125–135

Moore R, Wells W (2007) Private healthcare providers: A boost for NHS productivity? lournal of Management & Marketing in Healthcare 1(1):9– 12

Napel S (2003) Aspiration adaptation in the ultimatum minigame. Games and Economic Behavior 43:86–106

Nene D (2005) A beginners guide to Dependency Injection, URL http://www.theserverside.com/news/1321158/ A-beginners-guide-to-Dependency-Injection/, The Server Side online journal

NHS (2010) Choose and book, URL http://www.chooseandbook.nhs.uk/, last accessed June 2010

Pemantle R, Skyrms B (2000) A dynamic model of network formation. Proceedings of the National Academies of Science 97:9340–9346

Pemantle R, Skyrms B (2004) Network formation by reinforcement learning: the long and medium run. Mathematical Social Sciences 48(3):315–327

PostgreSQL Global Development Group (2010) PostgreSQL, URL http: //www.postgresql.org/, last accessed September 2010 Repast (2010) The Repast project, URL http://repast.sourceforge. net/repast_3, last accessed May 2010

Richiardi M (2003) The Promises and Perils of Agent-Based Computational Economics, Laboratorio R. Revelli, Centre for Employment Studies

Robertson R, Thorlby R (2010) Patient choice, The King’s Fund Briefing, 2008

Roth A, Erev I (1995) Learning in Extensive Form Games: Experimental Data and Simple Dynamic Models in the Intermediate Run. Games and Economic Behaviour 6

Royal College of General Practitioners (2006) Profile of UK General Practitioners 20006 (www.rcgp.org.uk/pdf/ISS INFO 01 JUL06.pdf), URL www.rcgp.org.uk/pdf/ISS_INFO_01_JUL06.pdf, last accessed March 2009

Rubinstein A (1998) Modeling Bounded Rationality. MIT Press

Rustichini A (1999) Optimal Properties of StimulusResponse Learning Models. Games and Economic Behavior 29:244–273

Ryan M (1994) Agency in health care: lessons for economists from sociologists. American Journal of Economics and Sociology 53:207–218

Sandia Labs (2010) The Jess Expert System Shell, URL http://herzberg. ca.sandia.gov/jess/, last accessed June 2010

Sarin R, Vahid F (1999) Payoff Assessments without Probabilities: A Simple Dynamic Model of Choice. Games and Economic Behavior 28:294–309

Sarin R, Vahid F (2001) Predicting How People Play Games: A Simple Dynamic Model of Choice. Games and Economic Behavior 34:104–122

Schelling T (1971) Dynamic Models of Segregation. Journal of Mathematical Sociology 1:143–186

Schuster S (2010) BRA: An Algorithm for Simulating Bounded Rational Agents. Computational Economics pp 1–19, URL http://dx.doi.org/ 10.1007/s10614-010-9231-1, 10.1007/s10614-010-9231-1 BIBLIOGRAPHY 213

Scott A (2000) Economics of General Practice. In: Cuyler A, Newhouse J (eds) Handbook of Health Economics, vol 1, Elsevier Science, Amsterdam, chap 22, pp 1175–2000

Scott A (2005) For love or money? Alternative methods of paying physicians, Paper presented to conference ’Sustaining Prosperity: New Reform Opportunities for Australia’, Melbourne

Scott A, Hall J (1995) Evaluating the effects of GP remuneration: problems and prospects. Health Policy 31:183–195

Sigaud O, Wilson S (2007) Learning classifier systems: A survey. Soft Computing 11:1065–1078

Simon H (1956a) A Behavioural Model of Rational Choice. In: Simon H (ed) Models of man, social and rational: mathematical essays on rational human behavior in a social setting, Wiley, New York

Simon H (1956b) Rational Choice and the Structure of the Environment. In: Simon H (ed) Models of man, social and rational: mathematical essays on rational human behavior in a social setting, Wiley, New York

Simon H (2000) Bounded Rationality: Today and Tomorrow. Mind and Society 1(1):25–39

Slikker M, van den Nouweland A (2000) Network formation models with costs for establishing links. Review of Economic Design 5:333–362

Smith JM (1982) Evolution and the Theory of Games. Cambridge University Press, Ca

Stahl DO (2000) Rule Learning in Symmetric Normal-Form Games: Theory and Evidence. Games and Economic Behavior 32:105–138

Sun R, Naveh I (2007) Social institution, cognition, and survival: A cognitive-social simulation. Mind and Society 6(2):15–42

Sun R, Slusarz P (2005) The Interaction of the Explicit and the Implicit in Skill Learning: A Dual-Process Approach. Psychological Review 112(1):159–192

Sun Corporation (2010a) Enterprise Java Beans, URL http://java.sun. com/products/ejb/, last accessed June 2010

Sun Corporation (2010b) Getting started with Java Messaging Service, URL http://java.sun.com/developer/technicalArticles/ Ecommerce/jms/index.html, last accessed June 2010

Sun Corporation (2010c) Java Enterprise Edition, URL http://java.sun. com/javaee/index.jsp, last accessed June 2010

Sun Corporation (2010d) Remote Method Invocation (RMI), URL http: //java.sun.com/javase/technologies/core/basic/rmi/index.jsp, last accessed June 2010

Sutton R, Barto A (1998) Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA

Swarm (2010) The Swarm project, URL http://wiki.swarm.org, last accessed June 2007

Terracotta (2010) The Terracotta Framework, URL http://www. terracotta.org, last accessed March 2010

Tesfatsion L (2006) Agent-based computational economics: A constructive approach to economic theory. In: Tesfatsion L, Judd K (eds) Handbook of Computational Economics, 2, Elsevier, chap 16, pp 831–877

The JBoss Community (2010a) JBoss Application Server, URL http:// www.jboss.org/jbossas, last accessed June 2010

The JBoss Community (2010b) JBoss Cache, URL http://www.jboss. org/jbosscache, last accessed June 2010

The Repast Symphony Project (2010) Repast Symphony, URL http:// repast.sourceforge.net/, last accessed May 2010

Vick S, Scott A (1998) Agency in health care. Examining patients’ preferences for attributes of the doctor-patient relationship. Journal of Health Economics 17:587–605

Watts A (2001) A dynamic model of network formation. Games and Economic Behaviour 34:331–341

Watts A (2002) Non-myopic formation of circle networks. Economic Letters 74:277–282 Williams S, Calnan M (1991) Key determinants of consumer satisfaction with general practice. Family Practice 8:237–242

Winters L (2006) Health Impact Assessment of the Patient Choice Agenda, Observatory Report Series

Woodward R, Warren-Boulton F (1984) Considering the effects of financial incentives and professional ethics on ’appropriate’ medical care. Journal of Health Economics 3:223–237

Young HP (1993) The Evolution of Conventions. Econometrica 61(1):57–84

Young HP (2003) Strategic Learning and its limits. Oxford University Press Zweifel P, Breyer F, Kifmann M (2005) Health Economics. Springer, Berlin