Experimental Economics: Some Methodological Notes

Fiore, Annamaria (2009): Experimental Economics: Some Methodological Notes.

Abstract

The aim of this work is presenting in a self-contained paper some methodological aspects as they are received in the current experimental literature. The purpose has been to make a critical review of some very influential papers dealing with methodological issues. In other words, the idea is to have a single paper where people first approaching experimental economics can find summarised (some) of the most important methodological issues. In particular, the focus is on some methodological practises still debated in experimental literature, such as attainment of control in experimental settings, subject pool, incentive mechanisms, repeated trials and learning. The hope is that increasing awareness on some sharing methodologies will improve the robustness of results in this research field.

• Item Type: MPRA Paper
• Language: English
• Keywords: Experimental Economics; Methodology; Control; Incentives; Learning; Deception
• Subjects: B - History of Economic Thought, Methodology, and Heterodox Approaches > B4 - Economic Methodology > B40 - General; C - Mathematical and Quantitative Methods > C9 - Design of Experiments > C90 - General

References

Abbink, K., Selten, R., and Sadriech A., 1999, Money does not Induce Risk Neutral Behaviour, but Binary Lotteries do even Worse, Theory and Decision, 46, 213-252.

Arthur, W. B.,2006, Out-of-equilibrium economics and agent-based modelling, in K. Judd and L. Tesfatsion, eds. Handbook of Computational Economics, Vol. 2: Agent-Based Computational Economics, North-Holland Press.

Bardsley, N., 2005, Experimental Economics and the Artificiality of Alteration, Journal of Economic Methodology, 12(2), 239-251.

Bardsley, N., and Moffat, P., 2005, The Experimetrics of Public Goods: Inferring Motivations from Contributions. Discussion Paper, The Centre for Decision Research and Experimental Economics, School of Economics, University of Nottingham.

Beattie, J., and Loomes, G., 1997, The Impact of Incentives Upon Risky Choice Experiments, Journal of Risk and Uncertainty, 14, 149-162.

Becker, G.M., DeGroot M.H., and J. Marschak, 1964, Measuring Utility by a Single-Response Sequential Method, Behavioral Science, 9, 226-232.

Benz, M., and Meier, S., 2006, Do People Behave in Experiments as in the Field? Evidence from Donations. Working Paper, Institute for Empirical Research in Economics.

Bernasconi, M., Kirchkamp, O., and Paruolo, P., 2006, Do Fiscal Variables Affect Fiscal Expectations? Experiments with Real Worls and Lab Data. CRIEFF Discussion Paper, Centre for Research into Industry, Enterprise, Finance and Firm.

Binmore, K., 1999, Why Experiment in Economics?, Economic Journal, 109, F16-F24.

Bohm, P., Lindén, J., and Sonnegård, J., 1997, Eliciting Reservation Prices: Becker-DeGroot-Marschack Mechanism vs. Market, Economic Journal, 107, 1079-1089.

Bonetti, S., 1998a, Experimental Economics and Deception, Journal of Economic Psychology, 19, 377-395.

Bonetti., S., 1998b, Reply to Hey and Starmer and McDaniel, Journal of Economic Psychology, 19, 411-414.

Brunner, E., Domhof, S., and Langer, F., 2002, Nonparametric Analysis of Longitudinal Data in Factorial Experiments, Wiley, NY.

Burns, P., 1985, Experience and Decision Making: A Comparison of Students and Businessmen in a Simulated Progressive Auction, in V. Smith ed. Research in Experimental Economics, Vol. 3, Greenwich: JAI Press.

Camerer, C.F., 1995, Individual Decision Making, in A. Roth and J. Kagel, eds. Handbook of Experimental Economics, Princeton: Princeton University Press, 587-703.

Camerer, C.F., 1997, Rules for Experimenting in Psychology and Economics, and Why They Differ, in W. Albers, W. Güth, P. Hammertein, B. Moldovanu and E. von Damme, eds. Understanding Strategic Interaction. Essays in Honor of Reinhard Selten, Springer, 313-327.

Camerer, C.F., and Ho, T.H., 1998, Experienced Weighted Attraction Learning in Normal-Form Games, Econometrica, 67(4), 827-874.

Camerer, C.F., and Hogarth, R.M., 1999, The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework, Journal of Risk and Uncertainty, 19, 7-42.

Camerer, C.F., Loewenstein, G., and Weber, M., 1989, The Curse of Knowledge in Economic Settings: An Experimental Analysis, Journal of Political Economy, 97, 1232-1254.

Cameron, L., 1999, Raising the Stakes in the Ultimatum Game: Experimental Evidence from Indonesia, Economic Inquiry, 37, 47-59.

Carbone, E., 2005, Demographics and Behaviour, Experimental Economics, 8(3), 217-232.

Chamberlin, E.H., 1948, The Theory of Monopolistic Competition, Cambridge: Harvard University Press.

Conover, W.J., 1980, Practical Nonparametric Statistics, Wiley, NY.

Cooper, D.J., Kagel, J.H., Lo, W., Gu, Q.L., 1999, Gaming against Managers in Incentive Systems: Experimental Results with Chinese Students and Chinese Managers, American Economic Review, 89(4), 781-804.

Cox, L.C., Smith, V.L., and Walker, J.K., 1992, Theory and Misbehavior of First Price Auctions: Comment, American Economic Review, 82(5), 1392-1412.

Cubitt, R., 2005, Experiments and the Domain of Economic Theory, Journal of Economic Methodology, 12(2), 197-210.

Cubitt, R., Starmer, C., and Sugden, R., 1998, On the Validity of the Random Lottery Incentive System, Experimental Economics, 1(2), 115-138.

Dohmen, T., Falk, A., Huffman, D., Sunde, U., Schupp, J., and Wagner, G.G., 2005, Individual Risk Attitudes: New Evidence from a Large, Representative, Experimentally-Validated Survey. IZA Discussion Paper.

Davis, D.D., and Holt C.A., 1993, Experimental Economics, Princeton: Princeton University Press.

Duhem, P., 1953, Physical Theory and Experiment, in H. Feigl and M. Brodbeck eds. Reading in the Philosophy of Science, Appleton Century Crofts, 235-252.

Dyer, D., Kagel, J.H., and Levin, D., 1989, A Comparison of Naïve and Experienced Bidders in Common Value Offer Auctions: A Laboratory Analysis, Economic Journal, 99, 108-115.

Erev, I., and Roth, A.E., 1998, Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria, American Economic Review, 88(4), 848-881.

Falk, A., and Fehr, E., 2003, Why Labour Market Experiments?, Labour Economics, 10, 399-406.

Fehr, E., and Gächter, S., 2004, Fairness and Retaliation: The Economics of Reciprocity, in Camerer, CF., Loewenstein, G., and Rabin M., eds. Advances in Behavioral Economics. Princeton: Princeton University Press.

Fischbacher, U., 1999, z-Tree – Zurich Toolbox for Readymade Economic Experiment – Experimenter’s Manual. Working Paper, Institute for Empirical Research in Economics, University of Zurich.

Forsythe, R., Horowitz, J.L., Savin, N.E., and Sefton, M., 1994, Fairness in Simple Bargaining Experiments, Games and Economic Behavior, 6, 347-369.

Friedman, D., 1988, Experimental Methods: Points of Consensus and Points of Contention. Working Paper, University of California at Santa Cruz.

Friedman, D., 1992, Theory and Misbehavior of First Price Auctions: Comment, American Economic Review, 82(5), 1374-1378.

Friedman, D., and Cassar, A., 2004, Economics Lab: An Introduction to Experimental Economics, Routledge, London & NY.

Friedman, D., and Sunder S., 1994, Experimental Methods, Cambridge: Cambridge University Press.

Friedman, M., 1953, Essays in Positive Economics. Chicago: University of Chicago Press.

Gibbons, R., 1992, Game Theory for Applied Economists, Princeton University Press.

Gode, D.K., and Sunder, S., 1993, Allocative Efficiency of Markets with Zero Intelligence Traders, Journal of Political Economy, 101, 119-137.

Greiner, B., 2004, The Online Recruitment System ORSEE - A Guide for the Organization of Experiments in Economics. Discussion Papers on Strategic Interaction, Max Planck Institute of Economics.

Grether D., and Plott C.R., 1979, Economic Theory of Choice and the Preference Reversal Phenomenon, American Economic Review, 69(4), 623-638.

Guala, F., 2005a, Economics in the Lab: Completeness vs. Testability, Journal of Economic Methodology, 12(2), 185-196.

Guala, F., 2005b, The Methodology of Experimental Economics, Cambridge University Press.

Guala, F., A Short History of Experimental Economics. Unpublished paper, Department of Sociology and Philosophy, University of Exeter.

Guillén, P., and Veszteg, R.F., 2006, Subject Pool Bias in Economic Experiments. Working Paper, Departamento de Teorìa e Historia Econòmica, Universidad de Granada.

Güth, W., Schmittberger, R., and Schwarz,, B., 1982, An Experimental Analysis of Ultimatum Bargaining, Journal of Economic Behavior and Organization, 3, 367-388.

Hands, D. Wade, 2001, Reflection without Rules. Economic Methodology and Contemporary Science Theory, Cambridge: Cambridge University Press.

Harrison, G.W., 1989, Theory and Misbehavior of First Price Auctions, American Economic Review, 79(4), 749-762.

Harrison, G.W., 1992, Theory and Misbehavior of First Price Auctions: Reply, American Economic Review, 82(5), 1426-1443.

Harrison, G.W., 1994, Expected Utility Theory and the Experimentalists, Empirical Economics, 19, 223-253.

Harrison, G.W., and Johnson, L., 2005, Identifying Altruism in the Laboratory. UCF Economics Working Paper No. 04-04.

Harrison, G. W., Lau, M.I., and Rutström, E.E., 2005, Dynamic Consistency in Denmark: A Longitudinal Field Experiment. UCF Economics Working Paper No. 05-02.

Harrison, G.W., and List, J.A., 2004, Field Experiments, Journal of Economic Literature, 42, 1009-1055.

Hausman, D.M., 2005, ‘Testing’ Game Theory, Journal of Economic Methodology, 12(2), 211-224.

Hertwig, R., and Ortmann, A., 2001, Experimental Practises in Economics: A Methodological Challenge for Psychologists?, Behavioral and Brain Science, 24, 383-403.

Hey, J.D., 1991, Experiments in Economics, Oxford: Blackwell.

Hey, J.D., 1998, Experimental Economics and Deception: A Comment, Journal of Economic Psychology, 19, 397-401.

Hey, J.D., 2001, Does Repetition Improve Consistency?, Experimental Economics, 4(1), 5-54.

Hey, J.D., and Lee J., 2005, Do Subjects Separate (or Are They Sophisticated)?, Experimental Economics, 8(3) 233-265.

Hoeffler, S., and Ariely, D., 1999, Constructing Stable Preferences: A Look into Dimensions of Experience and Their Impact on Preference, Journal of Consumer Psychology, 8(2), 113-139.

Hoffman, E., McCabe, K., and Smith, V.L., 1996, Social Distance and Other-Regarding Behavior in Dictator Games, American Economic Review, 86(3), 653-660.

Hogarth, R.M., 2005, The Challenge of Representative Design in Psychology and Economics, Journal of Economic Methodology, 12(2), 253-264.

Holland, J.H., 1997, Emergence, from chaos to order, Addison-Wesley.

Holland, J.H., 1995, Adaptation in Natural and Artificial Systems, Univ. of Michigan Press.

Holt, C.A., 1986, Preference Reversal and the Independence Axiom, American Economic Review, 76(3), 508-515.

Holt, C.A., and Laury, S.K., 2002, Risk Aversion and Incentive Effects, American Economic Review, 92(5), 1644-1655.

Isaac, M.R., and Walker, J.M., 1991, Costly Communication: An Experiment in a Nested Public Goods Problem, in T.R. Palfrey, ed. Laboratory Research in Political Economy, Ann Arbor: University of Michigan Press.

Jones, M.K., and Seaman, P.T., 2003, Sampling Bias in Economics Experiment: An Analysis of the Effect of Attrition among Subjects who Sign Up for Experiments. Discussion Paper, Economic Studies, University of Dundee.

Jones, S.R.G., 1992, Was There a Hawthorne Effect?, American Journal of Sociology, 98(3), 451-468.

Kagel, J.H., Battalio, R.C., and Walker, J.M., 1979, Volunteer Artifacts in Experiments in Economics: Specification of the Problem and Some Initial Data from a Small-Scale Field Experiment, in V. Smith ed. Research in Experimental Economics, Vol. 1, Greenwich: JAI Press.

Kagel, J.H., and Roth A.E., 1992, Theory and Misbehavior of First Price Auctions: Comment, American Economic Review, 82(5), 1379-1391.

Kagel, J.H., and Roth A.E., 1995, Handbook of Experimental Economics, Princeton: Princeton University Press.

Kahneman, D., and Tversky, A., 1979, Prospect Theory: An Analysis of Decision Under Risk, Econometrica, 47(2), 263-291.

Kim, O., Walker, J.M., 1984, The Free-Rider Problem: Experimental Evidence, Public Choice, 43, 3-24.

Kirman, A.P., 1992, Whom or What Does the Representative Individual Represent?, Journal of Economic Perspectives, 6(2), 117-136.

Knetsch, J.L., 1989, The Endowment Effect and Evidence of Nonreversible Indifference Curves, American Economic Review, 79(5), 1277-1284.

Landsberger, H.A., 1958, Hawthorne Revisited, Ithaca, NY: Cornell University.

Ledyard, J., 1995, Public Goods: A Survey of Experimental Research, in A. Roth and J. Kagel, eds. Handbook of Experimental Economics, Princeton: Princeton University Press, 111-194.

Levitt, S.D., and List, J.A., 2007, What Do Laboratory Experiments Measuring Social Preferences Reveal About the Real World?, Journal of Economic Perspectives, 21(2), 153-174.

Lichtenstein, S., and Slovic, P., 1971, Reversal of Preferences between Bids and Choices in Gambling Decisions, Journal of Experimental Psychology, 89, 46-55.

Lichtenstein, S., and Slovic, P., 1973, Response-Induced Reversals of Preferences in Gambling: An Extended Replication in Las Vegas, Journal of Experimental Psychology, 101, 16-20.

Lipsey, R.G., 1979, An Introduction to Positive Economics, 5th ed, London: Weidenfeld and Nicholson.

Loewenstein, G., 1999, Experimental Economics from the Vantage-Point of Behavioural Economics, Economic Journal, 109, F25-F34.

Loomes, G., 1999a, Some Lessons from Past Experiments and Some Challenges for the Future, Economic Journal, 109, F35-F45.

Loomes, G., 1999b, Experimental Economics: Introduction, Economic Journal, 109, F1-F4.

Mäki, U., 2005, Models are Experiments, Experiments are Model, Journal of Economic Methodology, 12(2), 303-316.

McCloskey, D.N., 1985, The Rhetoric of Economics, Madison: University of Wisconsin Press.

McDaniel, T., and Starmer, C., 1998, Experimental Economics and Deception, Journal of Economic Psychology, 19, 403-409.

Merlo, A., and Schotter, A., 1992, Theory and Misbehavior of First Price Auctions: Comment, American Economic Review, 82(5), 1413-1425.

Mill, J.S., 1874, On the Definition of Political Economy and the Method of Investigation Proper to It, in Essays on Some Unsettled Questions of Political Economy, 2nd ed. London: Longmans, Green, Reader, and Dyer [1st edition 1844].

Morgan, M.S., 2005, Experiment versus Models: New Phenomena, Inference and Surprise, Journal of Economic Methodology, 12(2), 317-329.

Ortmann, A., and Hertwig R., 2002, The Cost of Deception: Evidence from Psychology, Experimental Economics, 5(2), 111-131.

Ortmann, A., and Tichy, L., 1999, Understanding Gender Effects in the Laboratory: Evidence from Prisoner’s Dilemma Games, Journal of Economic Behavior and Organization, 39, 327-339.

Palfrey, T., and Porter R., 1991, Guidelines for Submission of Manuscripts on Experimental Economics, Econometrica, 59(4), 1197-1198.

Plott, C.R., 1982, Industrial Organization Theory and Experimental Economics, Journal of Economic Literature, 20, 1485-1527.

Plott, C.R., 1991, Will Economics Become an Experimental Science?, Southern Economic Journal, 57, 901-919.

Plott, C.R., 1996, Rational Individual Behaviour in Markets and Social Choice Processes: the Discovered Preference Hypothesis, in K. Arrow, M. Perlman, and C. Schmidt eds. Rational Foundations of Economic Behavior, London: MacMillan, 225-250.

Quine, W.V., 1953, From a Logical Point of View, Harvard University Press.

Read, D., 2005, Monetary Incentives, What Are They Good For?, Journal of Economic Methodology, 12(2), 265-276.

Read, D., Loewenstein, G., and Rabin, M., 1999, Choice Bracketing, Journal of Risk and Uncertainty, 19 (1-3), 171-197.

Roth, A.E., 1988, Laboratory Experimentation in Economics: A Methodological Overview, Economic Journal, 98, 974-1031.

Roth, A.E., 1995, Introduction to Experimental Economics, in A. Roth and J. Kagel, eds. Handbook of Experimental Economics, Princeton: Princeton University Press, 3-109.

Roth, A.E., and Erev, I., 1995, Learning in Extensive-Form Games: Experimental Data and Simple Dynamic Models in the Intermediate Term, Games and Economic Behavior, Special Issue: Nobel Symposium, 8(1), 164-212.

Roth, A.E., and Malouf, M.W.K., 1979, Game Theoretic Models and the Role of Information in Bargaining, Psychological Review, 574-594.

Rousseas, S.W., and Hart A.G., 1951, Experimental Verification of a Composite Indifference Map, Journal of Political Economy, 59, 288-318.

Rubinstein, A., 2001, A Theorist’s View of Experiments, European Economic Review, 45, 615-628.

Samuelson, P.A., and Nordhaus W.D., 1985, Principles of Economics, 12th ed.

Schelling, T.C., 1957, Bargaining, Communication, and Limited War, Journal of Conflict Resolution, 1, 19-36.

Schram, A., 2005, Artificiality: The Tension between Internal and External Validity in Economic Experiments, Journal of Economic Methodology, 12(2), 225-237.

Siegel, S., and Fouraker, L.E., 1960, Bargaining and Group Decision Making, New York: McGraw-Hill.

Slovic, P., 1995, The Construction of Preference, American Psychologist, 50(5), 364-371.

Slovic, P., and Lichtenstein, S., 1968, Relative Importance of Probabilities and Payoffs in Risk Taking, Journal of Experimental Psychology Monograph Supplement, 3(2), 1-18.

Smith, A., 1776, Inquiry into the Nature and Causes of the Wealth of the Nations, Edinburgh.

Smith, V.L., 1962, An Experimental Study of Competitive Market Behavior, Journal of Political Economy, 70(2), 111 - 137.

Smith, V.L., 1976, Experimental Economics: Induced Value Theory, American Economic Review, 66(2), 274-279.

Smith, V.L., 1982, Microeconomic System as an Experimental Science, American Economic Review, 72(5), 923-955.

Smith, V.L., 1994, Economics in the Laboratory, Journal of Economic Perspective, 8(1), 113-131.

Smith, V.L., and Walker J.M., 1993, Monetary Rewards and Decision Costs in Experimental Economics, Economic Inquiry, 31, 245-261.

Starmer, C., 1999a, Experiments in Economics… Should We Trust the Dismal Scientists in White Coats?, Journal of Economic Methodology, 6(1), 1-30.

Starmer, C., 1999b, Experimental Economics: Hard Science or Wasteful Tinkering?, Economic Journal, 109, F5-F15.

Starmer, C., 2005, Normative Notions in Descriptive Dialogues, Journal of Economic Methodology, 12(2), 277-290.

Starmer, C., and Sugden, R., 1991, Does the Random Lottery Incentive System Elicit True Preferences? An Experimental Investigation, American Economic Review, 81(4), 971-978.

Sugden, R., 2000, Credible Words: The Status of Theoretical Models in Economics, Journal of Economic Methodology, 7(1), 1-31.

Sugden, R., 2005a, Experiment, Theory, World: A Symposium on the Role of Experiments in Economics, Journal of Economic Methodology, 12(2), 177-184.

Sugden, R., 2005b, Experiments as Exhibits and Experiments as tests, Journal of Economic Methodology, 12(2), 177-184.

Tesfatsion, L., 1997, How economists can get alife, in W. B. Arthur, S. Durlauf and D. Lane, eds. The economy as an evolving complex system, II, Addison-Wesley.

Tesfatsion, L., 2001, Introduction to the special issue on agent-based computational economics, Journal of Economic Dynamics and Control, 25, 281–293.

Tenorio, R., and Cason, T.N., 2002, To Spin or not to Spin? Natural and Laboratory Experiments from “The Price is Right”, Economic Journal, 112, 476, 170-195.

Thaler, R., 1987, The Psychology of Choice and the Assumptions of Economics, in E. Roth ed. Laboratory Experimentation in Economics. Six Points of Views, Cambridge University Press.

Thurstone, L.L., 1931, The Indifference Function, Journal of Social Psychology, 2, 139-167.

Tversky, A., and Kahneman, D., 1981, The Framing of Decisions and the Psychology of Choice, Science, Vol. 211. no. 4481, 453 – 458.

Vickrey, W., 1961, Counterspeculation, Auctions, and Competitive Sealed Tenders, Journal of Finance, 16(1), 8-37.

Wallis, W.A., and M. Friedman, 1942, The Empirical Derivation of Indifference Functions, in O. Lange, F. McIntyre, and T.O. Yntema eds. Studies in Mathematical Economics and Econometrics in Memory of Henry Schultz, Chicago, University of Chicago Press, 175-189.

Wilcox, N., 1993, Lottery Choice, Incentives, Complexity and Decision Time, Economic Journal, 103, 1397-1417.

Wilde, L., 1980, On the Use of Laboratory Experiments in Economics, in J. Pitt ed. The Philosophy of Economics, Dordrecht: Reidel.