Humans versus computer algorithms in repeated mixed strategy games

Spiliopoulos, Leonidas (2008): Humans versus computer algorithms in repeated mixed strategy games.

Abstract

This paper is concerned with the modeling of strategic change in humans’ behavior when facing different types of opponents. In order to implement this efficiently a mixed experimental setup was used where subjects played a game with a unique mixed strategy Nash equilibrium for 100 rounds against 3 preprogrammed computer algorithms (CAs) designed to exploit different modes of play. In this context, substituting human opponents with computer algorithms designed to exploit commonly occurring human behavior increases the experimental control of the researcher allowing for more powerful statistical tests. The results indicate that subjects significantly change their behavior conditional on the type of CA opponent, exhibiting within-sub jects heterogeneity, but that there exists comparatively little between-subjects heterogeneity since players seemed to follow very similar strategies against each algorithm. Simple heuristics, such as win-stay/lose-shift, were found to model subjects and make out of sample predictions as well as, if not better than, more complicated models such as individually estimated EWA learning models which suffered from overfitting. Subjects modified their strategies in the direction of better response as calculated from CA simulations of various learning models, albeit not perfectly. Examples include the observation that subjects randomized more effectively as the pattern recognition depth of the CAs increased, and the drastic reduction in the use of the win-stay/lose-shift heuristic when facing a CA designed to exploit this behavior.

• Item Type: MPRA Paper
• Institution: University of Sydney
• Language: English
• Keywords: Behavioral game theory; Learning; Experimental economics; Simulations; Experience weighted attraction learning; Simulations; Repeated games; Mixed Strategy Nash equilibria; Economics and psychology
• Subjects: C - Mathematical and Quantitative Methods > C9 - Design of Experiments; C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling > C63 - Computational Techniques ; Simulation Modeling; C - Mathematical and Quantitative Methods > C7 - Game Theory and Bargaining Theory > C70 - General; C - Mathematical and Quantitative Methods > C7 - Game Theory and Bargaining Theory > C73 - Stochastic and Dynamic Games ; Evolutionary Games ; Repeated Games; C - Mathematical and Quantitative Methods > C7 - Game Theory and Bargaining Theory > C72 - Noncooperative Games; C - Mathematical and Quantitative Methods > C9 - Design of Experiments > C91 - Laboratory, Individual Behavior

References

Axelrod, R. (1985). The Evolution of Cooperation. Basic Books. Barraclough, D. J., M. L. Conroy, and D. Lee (2004, April). Prefrontal cortex and decision making in a mixed strategy game. Nature Neuroscience 7 (4), 404–10. Blount, S. (1995, August). When social outcomes aren’t fair: The effect of casual attributions on preferences. Organizational Behavior and Human Decision Processes 63 (2), 131–44. Bonetti, S. (1998). Experimental economics and deception. Journal of Economic Psychology 19, 377–395. Budescu, D. V. and A. Rapoport (1994). Sub jective randomization in one- and two-person games. Journal of Behavioral Decision Making 7, 261–78. Camerer, C., G. Loewenstein, and D. Prelec (2005, March). Neuroeconomics: How neuroscience can inform economics. Journal of Economic Literature 43 (1), 9–64. Camerer, C. F. and T. Ho (1999). Experience-weighted attraction learning in normal-form games. Econometrica 67, 827–74. Coricelli, G. (2005). Strategic interaction in iterated zero-sum games. Homo Oeconomicus, forthcoming. Cosmides, L. and J. Tooby (1987). From evolution to behavior: Evolutionary psychology as the missing link. In J. Dupre (Ed.), The latest on the best: Essays on evolution and optimality. Cambridge, MA: MIT Press. Davis, D. and C. A. Holt (1992). Experimental Economics. Princeton University Press, Princeton, NJ. Dursch, P., A. Kolb, J. Oechssler, and B. C. Schipper (2005). Rage against the machines: How subjects learn to play against computers. Technical report, University of California, Davis. Efron, B. (1987). Better bootstrap confidence intervals. Journal of the American Statistical Association 82, 171–200. Efron, B. and R. Tibshirani (1994). An Introduction to the Bootstrap. Chapman & Hall/CRC. Eurostat (2006, 13 July). Minimum wages in the eu25. Fisher, R. (1920). A mathematical examination of the methods of determining the accuracy of observation by the mean error and the mean square error. Monthly Notes of the Royal Astronomical Society 80, 758–770. Fox, J. (1972). The learning of strategies in a simple, two-person zero-sum game without saddlepoint. Behavioral Science 17, 300–308. Gigerenzer, G. (2000). Adaptive thinking: Rationality in the real world. New York: Oxford University Press. Gigerenzer, G. and R. Selten (Eds.) (2001). Bounded rationality: The adaptive toolbox. Cambridge, MA: MIT Press. Gilovich, T., D. Griffin, and D. E. Kahneman (2002). Heuristics and biases: The psychology of intuitive judgment. Cambridge, UK: Cambridge University Press. Gorard, S. (2005). The advantages of the mean deviation. British Journal of Educational Studies 53 (4), 417–30. Harrison, G. W. (1989). Theory and misbehavior of first-price auctions. American Economic Review 79 (4), 749–62. Hertwig, R. and A. Ortmann (2001). Experimental practices in economics: A methodological challenge for psychologists? Behavioral and Brain Sciences 24, 383–451. Huber, P. (1981). Robust Statistics. New York, John Wiley and Sons. Huber, P. J. (1967). The behavior of maximum likelihood estimates under nonstandard conditions. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1, Berkeley, CA, pp. 221–223. University of California Press. Matheson, I. (1990). A critical comparison of least absolute deviation fitting(robust) and least squares fittings: the importance of error distributions. Computers & chemistry 14 (1), 49–57. Matlab (2007). Mathworks, Inc., Natick, MA. Messick, D. M. (1967). Interdependent decision strategies in zero-sum games: A computer-controlled study. Behavioral Science 12, 33–48. Mitchell, M. (1999). An Introduction to Genetic Algorithms (Fifth ed.). The MIT Press. Mitropoulos, A. (2001). On the measurement of the predictive success of learning theories in repeated games. Economics Working Paper Archive EconWPA. Nelder, J. A. and R. Mead (1965). A simplex method for function minimization. Computer Journal 7, 308–313. Nowak, M. and K. Sigmund (1993, July). A strategy of win-stay, lose-shift that outperforms tit-for-tat in the prisoner’s dilemma game. Nature 364, 56–58. Nyarko, Y. and A. Schotter (2002). An experimental study of belief learning using elicited beliefs. Econometrica 70 (3), 971. O’Neill, B. (1987). Nonmetric test of the minimax theory of two-person zerosum games. In Proceedings of the National Academy of Sciences, Volume 84, pp. 2106–9. Rapoport, A. and D. Budescu (1997). Randomization in individual choice behavior. Psychological Review 104 (603-617). Shachat, J. and T. J. Swarthout (2002). Learning about learning in games through experimental control of strategic interdependence strategic interdependence. Experimental 0310003, EconWPA. Shachat, J. and T. J. Swarthout (2004). Do we detect and exploit mixed strategy play by opponents? Mathematical Methods of Operations Research 59 (3), 359–373. Sidak, Z. (1967). Rectangular confidence regions for the means of multivariate normal distributions. Journal of the American Statistical Association 62, 626–633. Siebrasse, N. (2000). Generalized win-stay, lose-shift is robust in the repeated prisoners’ dilemma with noise played by multi-state automata. Smith, V. L. and J. M. Walker (1993, April). Rewards, experience and decision costs in first price auctions. Economic Inquiry 31 (2), 237–245. Spiliopoulos, L. (2008). Do repeated players detect patterns in opponents? revisiting the Nyarko and Schotter belief elicitation experiment. StataCorp (2007). Stata Statistical Software: Release 10. College Station, TX: StataCorp LP. Svenson, O. (1981). Are we all less risky and more skillful than our fellow drivers? Acta Psychologica 47, 143–48. Tesfatsion, L. and K. L. Judd (2006). Handbook of Computational Economics Volume 2. Elsevier/North-Holland (Handbooks in Economics Series). Walker, J. M., V. L. Smith, and J. C. Cox (1987). Bidding behavior in first-price sealed-bid auctions: Use of computerized nash competitors. Economics Letters 23 (3), 239–244. White, H. (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica 48, 817–830. Wilson, H. (1978). Least squares versus minimum absolute deviations estimation in linear models. Decision Sciences 9, 322–335.