Incentives and Payment Mechanisms in Preference Elicitation

Drichoutis, Andreas C. and Palma, Marco and Feldman, Paul (2024): Incentives and Payment Mechanisms in Preference Elicitation.

Abstract

Previous literature analyzing the effects of incentive compatibility of experimental payment mechanisms is dominated by theory. With overwhelming evidence of theory violations in a multiplicity of domains, we fill this gap by empirically exploring the effects of different payment mechanisms in induced preference elicitation using a large sample of over 3800 participants across three experiments. In Experiment 1, we collected responses for offer prices to sell a card like in Cason and Plott (2014), systematically varying on a between-subjects basis the way subjects received payments over repeated rounds, by either paying for all decisions (and various modifications) or just one, as well as making the payments certain, probabilistic or purely hypothetical. While we find that the magnitude of the induced value and the range of the prices used to draw a random price significantly affect misbidding behavior, neither the payment mechanism nor the certainty of payment affected misbidding. In Experiment 2, we replaced the BDM mechanism with a second price auction and found similar results, albeit less misbidding rates. In Experiment 3, we examine the effect of payment mechanisms on choice under risk and find portfolio effects (i.e., paying all rounds) when the lottery pairs do not involve options with certainty. Overall, our empirical exercise shows that payment mechanism design considerations should place more weight on the choice architecture rather than on incentive compatibility.

• Item Type: MPRA Paper
• Language: English
• Keywords: Becker-DeGroot-Marschak mechanism, second price auction, risk choices, preference elicitation, choice architecture
• Subjects: C - Mathematical and Quantitative Methods > C8 - Data Collection and Data Estimation Methodology ; Computer Programs > C80 - General; C - Mathematical and Quantitative Methods > C9 - Design of Experiments > C91 - Laboratory, Individual Behavior; D - Microeconomics > D4 - Market Structure, Pricing, and Design > D44 - Auctions

References

Agranov, M. and P. Ortoleva (2017). Stochastic choice and preferences for randomization. Journal of Political Economy 125 (1), 40–68.

Ahler, D. J., C. E. Roush, and G. Sood (2021). The micro-task market for lemons: data quality on amazon’s mechanical turk. Political Science Research and Methods, 1–20.

Ahles, A., M. A. Palma, and A. C. Drichoutis (2024). Testing the effectiveness of lottery incentives in online experiments. American Journal of Agricultural Economicsc (forthcoming).

Allais, M. (1953). Le comportement de l'homme rationnel devant le risque: critique des postulats et axiomes de l’´ecole am´ericaine. Econometrica: journal of the Econometric Society, 503–546.

Azrieli, Y., C. P. Chambers, and P. J. Healy (2018). Incentives in experiments: A theoretical analysis. Journal of Political Economy 126 (4), 1472–1503.

Azrieli, Y., C. P. Chambers, and P. J. Healy (2020). Incentives in experiments with objective lotteries. Experimental Economics 23 (1), 1–29.

Baltussen, G., G. T. Post, M. J. van den Assem, and P. P. Wakker. Random incentive systems in a dynamic choice experiment. 15 (3), 418–443.

Beattie, J. and G. Loomes (1997). The impact of incentives upon risky choice experiments. Journal of Risk and Uncertainty 14 (2), 155–168.

Benjamin, D. J., M. A. Fontana, and M. S. Kimball (2020). Reconsidering risk aversion. Technical report, National Bureau of Economic Research.

Blavatskyy, P., A. Ortmann, and V. Panchenko (2022, February). On the experimental robustness of the allais paradox. American Economic Journal: Microeconomics 14 (1), 143–63.

Bolle, F. High reward experiments without high expenditure for the experimenter? 11 (2), 157–167.

Bra˜nas Garza, P., D. Jorrat, A. M. Esp´ın, et al. (2023). Paid and hypothetical time preferences are the same: lab, field and online evidence. Experimental Economics 26, 412–434.

Breig, Z. and P. Feldman (2023). Revealing risky mistakes through revisions. Available at SSRN 3975829 .

Briz, T., A. C. Drichoutis, and R. M. Nayga Jr (2017). Randomization to treatment failure in experimental auctions: The value of data from training rounds. Journal of Behavioral and Experimental Economics 71, 56–66.

Brown, A. L. and P. J. Healy (2018). Separated decisions. European Economic Review 101, 20–34.

Brown, A. L., J. Liu, and M. Tsoi (2023, June 12). Is there a better way to elicit valuations than the BDM?

Buschena, D. and D. Zilberman (2000). Generalized expected utility, heteroscedastic error, and path dependence in risky choice. Journal of Risk and Uncertainty 20, 67–88.

Cason, T. N. and C. R. Plott (2014). Misconceptions and game form recognition: Challenges to theories of revealed preference and framing. Journal of Political Economy 122 (6), 1235–1270.

Chandler, J., C. Rosenzweig, A. J. Moss, J. Robinson, and L. Litman (2019). Online panels in social science research: Expanding sampling methods beyond mechanical Turk. Behavior Research Methods 51, 2022–2038.

Charness, G., C. Eckel, U. Gneezy, and A. Kajackaite (2018). Complexity in risk elicitation may affect the conclusions: A demonstration using gender differences. Journal of risk and uncertainty 56, 1–17.

Charness, G., U. Gneezy, and B. Halladay (2016). Experimental methods: Pay one or pay all. Journal of Economic Behavior & Organization 131, 141–150.

Chmielewski, M. and S. C. Kucker (2020). An MTurk crisis? shifts in data quality and the impact on study results. Social Psychological and Personality Science 11 (4), 464–473.

Clot, S., G. Grolleau, and L. Ibanez. Shall we pay all? an experimental test of random incentivized systems. 73, 93–98.

Cochran, W. G. and D. B. Rubin (1973). Controlling bias in observational studies: A review. Sankhy¯a: The Indian Journal of Statistics, Series A 35 (4), 417–446.

Cox, J. C., V. Sadiraj, and U. Schmidt (2015). Paradoxes and mechanisms for choice under risk. Experimental Economics 18, 215–250.

Cubitt, R. P., C. Starmer, and R. Sugden (1998). On the validity of the random lottery incentive system. Experimental Economics 1, 115–131.

Danz, D., L. Vesterlund, and A. J. Wilson (2022). Belief elicitation and behavioral incentive compatibility. American Economic Review 112 (9), 2851–2883.

Deaton, A. and N. Cartwright (2018). Understanding and misunderstanding randomized controlled trials. Social Science & Medicine 210, 2–21.

Diggle, P. J., P. Heagerty, K.-Y. Liang, and S. L. Zeger (2002). Analysis of Longitudinal Data (2nd ed.). New York, USA: Oxford University Press Inc.

Drichoutis, A. C. and R. M. Nayga Jr (2022). Game form recognition in preference elicitation, cognitive abilities, and cognitive load. Journal of Economic Behavior & Organization 193, 49–65.

Dwenger, N., D. Kubler, and G. Weizsacker (2018). Flipping a coin: Evidence from university applications. Journal of Public Economics 167, 240–250.

Enke, B., U. Gneezy, B. Hall, D. Martin, V. Nelidov, T. Offerman, and J. van de Ven (2023, 07). Cognitive Biases: Mistakes or Missing Stakes? The Review of Economics and Statistics 105 (4), 818–832.

Feldman, P. and J. Rehbeck (2022). Revealing a preference for mixtures: An experimental study of risk. Quantitative Economics 13 (2), 761–786.

Feldman, P. J. and P. J. Ferraro (2023). A certainty effect for preference reversals under risk: Experiment and theory. Working paper.

Freeman, D. J. and G. Mayraz (2019). Why choice lists increase risk taking. Experimental Economics 22, 131–154.

Gneezy, U., A. Imas, and J. List (2015, February). Estimating individual ambiguity aversion: A simple approach. Working Paper 20982, National Bureau of Economic Research.

Grether, D. M. and C. R. Plott (1979). Economic theory of choice and the preference reversal phenomenon. The american economic review 69 (4), 623–638.

Haaland, I., C. Roth, and J. Wohlfart (2023, March). Designing information provision experiments. Journal of Economic Literature 61 (1), 3–40.

Hackethal, A., M. Kirchler, C. Laudenbach, M. Razen, and A. Weber (2023). On the role of monetary incentives in risk preference elicitation experiments. Journal of Risk and Uncertainty 66, 189–213.

Ham, J. C., J. H. Kagel, and S. F. Lehrer (2005). Randomization, endogeneity and laboratory experiments: the role of cash balances in private value auctions. Journal of Econometrics 125 (1), 175–205. Experimental and non-experimental evaluation of economic policy and models.

Hansen, R. G. and J. R. Lott (1991). The winner's curse and public information in common value auctions: Comment. The American Economic Review 81 (1), 347–361.

Harrison, G. W. (1989). Theory and misbehavior of first-price auctions. The American Economic Review, 749–762.

Harrison, G. W. (1992). Theory and misbehavior of first-price auctions: Reply. The American Economic Review 82 (5), 1426–1443.

Harrison, G. W. and J. T. Swarthout (2014). Experimental payment protocols and the bipolar behaviorist. Theory and Decision 77, 423–438. 31

Ho, D. E., K. Imai, G. King, and E. A. Stuart (2007). Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis 15 (3), 199–236.

Holt, C. A. (1986). Preference reversals and the independence axiom. The American Economic Review 76 (3), 508–515.

Holt, C. A. and S. K. Laury (2002). Risk aversion and incentive effects. American Economic Review 92 (5), 1644–1655.

Imbens, G. W. and D. B. Rubin (2016). Causal Inference for Statistics, Social, and Biomedical Sciences, An introduction. Cambridge and New York: Cambridge University Press.

Imbens, G. W. and J. M. Wooldridge (2009). Recent developments in the econometrics of program evaluation. Journal of Economic Literature 47 (1), 5–86.

Irwin, J. R., G. H. McClelland, and W. D. Schulze (1992). Hypothetical and real consequences in experimental auctions for insurance against low-probability risks. Journal of Behavioral Decision Making 5 (2), 107–116.

Johnson, C., A. Baillon, H. Bleichrodt, Z. Li, D. van Dolder, and P. P. Wakker (2021). PRINCE: An improved method for measuring incentivized preferences. Journal of Risk and Uncertainty 62, 1–28.

Kagel, J. H. and D. Levin (1986). The winner’s curse and public information in common value auctions. The American Economic Review 76 (5), 894–920.

Kagel, J. H. and D. Levin (1991). The winner’s curse and public information in common value auctions: Reply. The American Economic Review 81 (1), 362–369.

Karni, E. and Z. Safra (1987). “preference reversal” and the observability of preferences by experimental methods. Econometrica 55 (3), 675–685.

Kendall, C. and A. Chakraborty (2022). Future self-proof elicitation mechanisms. Available at SSRN: https: // ssrn. com/ abstract= 4032946 .

Kupper, L. L. and K. B. Hafner (1989). How appropriate are popular sample size formulas? The American Statistician 43 (2), 101–105.

Li, Y. (2021). The ABC mechanism: an incentive compatible payoff mechanism for elicitation of outcome and probability transformations. Experimental Economics 24, 1019–1046.

Li, Z., J. Muller, P. P. Wakker, and T. V. Wang (2017). The rich domain of ambiguity explored. Management Science 64 (7), 3227–3240.

Lichtenstein, S. and P. Slovic (1971). Reversals of preference between bids and choices in gambling decisions. Journal of experimental psychology 89 (1), 46.

Litman, L. and J. Robinson (2021). Beyond mechanical Turk: Using online market research platforms. In Conducting Online Research on Amazon Mechanical Turk and Beyond. 32

Liu, H. and T. Wu (2005). Sample size calculation and power analysis of time-averaged difference. Journal of Modern Applied Statistical Methods 4 (2), 434–445.

Machina, M. J. (1989). Dynamic consistency and non-expected utility models of choice under uncertainty. Journal of Economic Literature 27 (4), 1622–1668.

Mamadehussene, S. and F. Sguera (2022). On the reliability of the bdm mechanism. Management Science 69 (2), 1166–1179.

Martınez-Marquina, A., M. Niederle, and E. Vespa (2019). Failures in contingent reasoning: The role of uncertainty. American Economic Review 109 (10), 3437–3474.

McGranaghan, C., K. Nielsen, T. O’Donoghue, J. Somerville, and C. D. Sprenger (2024). Distinguishing common ratio preferences from common ratio effects using paired valuation tasks. American Economic Review 114 (2), 307–347.

Moher, D., S. Hopewell, K. F. Schulz, V. Montori, P. C. Gotzsche, P. J. Devereaux, D. Elbourne, M. Egger, and D. G. Altman (2010). CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. BMJ 340.

Molnar, A. (2019). SMARTRIQS: A simple method allowing real-time respondent interaction in qualtrics surveys. Journal of Behavioral and Experimental Finance 22, 161–169.

Mutz, D. C. and R. Pemantle (2015). Standards for experimental research: Encouraging a better understanding of experimental methods. Journal of Experimental Political Science 2 (2), 192–215.

Peer, E., L. Brandimarte, S. Samat, and A. Acquisti (2017). Beyond the Turk: Alternative platforms for crowdsourcing behavioral research. Journal of Experimental Social Psychology 70, 153–163.

Peer, E., D. Rothschild, A. Gordon, Z. Evernden, and E. Damer (2022). Data quality of platforms and panels for online behavioral research. Behavior Research Methods 54, 1643–1662.

Rabin, M. (2013). Risk aversion and expected-utility theory: A calibration theorem. In Handbook of the fundamentals of financial decision making: Part I, pp. 241–252. World Scientific.

Samuelson, P. A. (1938). A note on the pure theory of consumer's behaviour. Economica 5 (17), 61–71.

Segal, U. (1988). Does the preference reversal phenomenon necessarily contradict the independence axiom? The American Economic Review 78 (1), 233–236.

Serizawa, S., N. Shimada, and T. T. K. Tse (2024, February). Toward an understanding of dominated bidding in a vickrey auction experiment. Discussion Paper 1229, The Institute of Social and Economic Research, Osaka University, 6-1 Mihogaoka, Ibaraki, Osaka 567-0047, Japan. Revised April 2024.

Smith, V. L. (1976). Experimental economics: Induced value theory. The American Economic Review 66 (2), 274–279. 33

Stagnaro, M. N., J. Druckman, a. berinsky, A. A. Arechar, R. Willer, and D. G. Rand (2024, Feb). Representativeness versus attentiveness: A comparison across nine online survey samples.

Starmer, C. and R. Sugden (1991). Does the random-lottery incentive system elicit true preferences? an experimental investigation. The American Economic Review 81 (4), 971–978.

Vassilopoulos, A., A. C. Drichoutis, and R. M. Nayga Jr (2018). Loss aversion, expectations and anchoring in the BDM mechanism. Munich Personal RePEc Archive No. 85635