How to run an experimental auction: A review of recent advances

Canavari, Maurizio and Drichoutis, Andreas C. and Lusk, Jayson L. and Nayga, Rodolfo (2018): How to run an experimental auction: A review of recent advances.

Abstract

In this paper, we review the recent advances in the literature of experimental auctions and provide practical advice and guidelines for researchers. We focus on issues related to randomization to treatment and causal identification of treatment effects, on design issues such as selection between different elicitation formats, multiple auction groups in a single session, and house money effects. We also discuss sample size issues related to recent trends about pre-registration and pre-analysis plans. We then present the pros and cons of moving auction studies from the lab to the field and review the recent literature on behavioral factors that have been identified as important for auction outcomes.

• Item Type: MPRA Paper
• Language: English
• Keywords: auctions; randomization; sample size; pre-registration; field experiments; behavioral factors
• Subjects: C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C57 - Econometrics of Games and Auctions; C - Mathematical and Quantitative Methods > C9 - Design of Experiments > C90 - General; D - Microeconomics > D4 - Market Structure, Pricing, and Design > D44 - Auctions

References

Abbink, K. and H. Hennig-Schmidt (2006). Neutral versus loaded instructions in a bribery experiment. Experimental Economics 9 (2), 103-121.

Abeler, J., A. Falk, L. Goette, and D. Huffman (2011). Reference points and effort provision. American Economic Review 101 (2), 470-92.

Akaichi, F., J. R. M. Nayga, and L. L. Nalley (2017). Are there trade-offs in valuation with respect to greenhouse gas emissions, origin and food miles attributes? European Review of Agricultural Economics 44 (1), 3-31.

Alfnes, F. (2009). Valuing product attributes in vickrey auctions when market substitutes are available. European Review of Agricultural Economics 36 (2), 133-149.

Alfnes, F., A. G. Guttormsen, G. Steine, and K. Kolstad (2006). Consumers' willingness to pay for the color of salmon: A choice experiment with real economic incentives. American Journal of Agricultural Economics 88 (4), 1050-1061.

Alfnes, F. and K. Rickertsen (2007). Extrapolating experimental-auction results using a stated choice survey. European Review of Agricultural Economics 34 (3), 345-363.

Altman, D. G. (1985). Comparability of randomised groups. Journal of the Royal Statistical Society. Series D (The Statistician) 34 (1), 125-136.

Andersen, S., G. W. Harrison, M. I. Lau, and E. E. Rutstrom (2006). Elicitation using multiple price list formats. Experimental Economics 9 (4), 383-405.

Andersen, S., G. W. Harrison, M. I. Lau, and E. E. Rutstrom (2008). Lost in state space: Are preferences stable? International Economic Review 49 (3), 1091-1112.

Anderson, M. L. and J. Magruder (2017). Split-sample strategies for avoiding false discoveries. National Bureau of Economic Research Working Paper Series No. 23544.

Andreoni, J., Y.-K. Che, and J. Kim (2007). Asymmetric information about rivals' types in standard auctions: An experiment. Games and Economic Behavior 59 (2), 240-259.

Arechar, A. A., S. Gachter, and L. Molleman (2018). Conducting interactive experiments online. Experimental Economics 21 (1), 99-131.

Ariely, D., G. Loewenstein, and D. Prelec (2003). "coherent arbitrariness": Stable demand curves without stable preferences. Quarterly Journal of Economics 118 (1), 73-105.

Augenblick, N. (2016). The sunk-cost fallacy in penny auctions. The Review of Economic Studies 83 (1), 58-86.

Austin, P. C. (2009, November). Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Statistics in Medicine 28 (25), 3083-3107.

Ausubel, L. M. (2004). An efficient ascending-bid auction for multiple objects. American Economic Review 94 (5), 1452-1475.

Banerji, A. and N. Gupta (2014). Detection, identiffication, and estimation of loss aversion: Evidence from an auction experiment. American Economic Journal: Microeconomics 6 (1), 91-133.

Bellemare, C., L. Bissonnette, and S. Kroger (2016, Nov). Simulating power of economic experiments: the powerbbk package. Journal of the Economic Science Association 2 (2), 157-168.

Belot, M. and J. James (2014). A new perspective on the issue of selection bias in randomized controlled field experiments. Economics Letters 124 (3), 326-328.

Belton, C. A. and R. Sugden (2018). Attention and novelty: An experimental investigation of order effects in multiple valuation tasks. Journal of Economic Psychology 67, 103-115.

Benjamin, D. J., J. O. Berger, M. Johannesson, B. A. Nosek, E. J. Wagenmakers, R. Berk, K. A. Bollen, B. Brembs, L. Brown, C. Camerer, D. Cesarini, C. D. Chambers, M. Clyde, T. D. Cook, P. De Boeck, Z. Dienes, A. Dreber, K. Easwaran, C. Efferson, E. Fehr, F. Fidler, A. P. Field, M. Forster, E. I. George, R. Gonzalez, S. Goodman, E. Green, D. P. Green, A. G. Greenwald, J. D. Hadfield, L. V. Hedges, L. Held, T. Hua Ho, H. Hoijtink, D. J. Hruschka, K. Imai, G. Imbens, J. P. A. Ioannidis, M. Jeon, J. H. Jones, M. Kirchler, D. Laibson, J. List, R. Little, A. Lupia, E. Machery, S. E. Maxwell, M. McCarthy, D. A. Moore, S. L. Morgan, M. Munafo, S. Nakagawa, B. Nyhan, T. H. Parker, L. Pericchi, M. Perugini, J. Rouder, J. Rousseau, V. Savalei, F. D. Schonbrodt, T. Sellke, B. Sinclair, D. Tingley, T. Van Zandt, S. Vazire, D. J. Watts, C. Winship, R. L. Wolpert, Y. Xie, C. Young, J. Zinman, and V. E. Johnson (2018). Redefine statistical significance. Nature Human Behaviour 2 (1), 6-10.

Bi, X., L. House, Z. Gao, and F. Gmitter (2012). Sensory evaluation and experimental auctions: Measuring willingness to pay for specific sensory attributes. American Journal of Agricultural Economics 94 (2), 562-568.

Borghans, L., A. L. Duckworth, J. J. Heckman, and B. t. Weel (2008). The economics and psychology of personality traits. Journal of Human Resources 43 (4), 972-1059.

Briz, T., A. C. Drichoutis, and R. M. Nayga Jr (2017). Randomization to treatment failure in experimental auctions: The value of data from training rounds. Journal of Behavioral and Experimental Economics 71, 56-66.

Brodeur, A., N. Cook, and A. Heyes (2018). Methods matter: P-hacking and causal inference in economics. IZA Discussion Paper No. 11796 . Brodeur, A., M. Le, M. Sangnier, and Y. Zylberberg (2016). Star wars: The empirics strike back. American Economic Journal: Applied Economics 8 (1), 1-32.

Brooks, K. and J. L. Lusk (2010). Stated and revealed preferences for organic and cloned milk: Combining choice experiment and scanner data. American Journal of Agricultural Economics 92 (4), 1229-1241.

Burke, W. J. (2009). Fitting and interpreting cragg's tobit alternative using stata. Stata Journal 9 (4), 584-592.

Bushong, B., L. M. King, C. F. Camerer, and A. Rangel (2010). Pavlovian processes in consumer choice: The physical presence of a good increases willingness-to-pay. American Economic Review 100 (4), 1556-71.

Button, K. S., J. P. A. Ioannidis, C. Mokrysz, B. A. Nosek, J. Flint, E. S. J. Robinson, and M. R. Munafo (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience 14, 365.

Camerer, C. F., A. Dreber, E. Forsell, T.-H. Ho, J. Huber, M. Johannesson, M. Kirchler, J. Almenberg, A. Altmejd, T. Chan, E. Heikensten, F. Holzmeister, T. Imai, S. Isaksson, G. Nave, T. Pfeiffer, M. Razen, and H. Wu (2016). Evaluating replicability of laboratory experiments in economics. Science 351 (6280), 1433-1436.

Camerer, C. F., A. Dreber, F. Holzmeister, T.-H. Ho, J. Huber, M. Johannesson, M. Kirchler, G. Nave, B. A. Nosek, T. Pfeiffer, A. Altmejd, N. Buttrick, T. Chan, Y. Chen, E. Forsell, A. Gampa, E. Heikensten, L. Hummer, T. Imai, S. Isaksson, D. Manfredi, J. Rose, E.-J. Wagenmakers, and H. Wu (2018). Evaluating the replicability of social science experiments in nature and science between 2010 and 2015. Nature Human Behaviour 2 (9), 637-644.

Capra, M. C. (2004). Mood-driven behavior in strategic interactions. American Economic Review 94 (2), 367-372.

Capra, M. C., S. Meer, and K. Lanier (2010). The effects of induced mood on bidding in random nth-price auctions. Journal of Economic Behavior & Organization 75 (2), 223-234.

Casari, M., J. C. Ham, and J. H. Kagel (2007). Selection bias, demographic effects, and ability effects in common value auction experiments. American Economic Review 97 (4), 1278-1304.

Cason, T. N. and C. R. Plott (2014). Misconceptions and game form recognition: Challenges to theories of revealed preference and framing. Journal of Political Economy 122 (6), 1235-1270.

Chambers, C. D. (2018). Introducing the transparency and openness promotion (top) guidelines and badges for open practices at cortex. Cortex 106, 316-318.

Chang, A. C. and P. Li (2017). A preanalysis plan to replicate sixty economics research papers that worked half of the time. American Economic Review 107 (5), 60-64.

Chang, A. C. and P. Li (2018). Is economics research replicable? Sixty published papers from thirteen journals say "often not". Critical Finance Review 7.

Chang, J. B., J. L. Lusk, and F. B. Norwood (2009). How closely do hypothetical surveys and laboratory experiments predict field behavior? American Journal of Agricultural Economics 91 (2), 518-534.

Chen, Y., P. Katuscak, and E. Ozdenoren (2009). Why can't a woman bid more like a man? Games and Economic Behavior 77 (1), 181-213.

Cherry, T. L., T. D. Crocker, and J. F. Shogren (2003). Rationality spillovers. Journal of Environmental Economics and Management 45 (1), 63-84.

Christensen, G. and E. Miguel (2018). Transparency, reproducibility, and the credibility of economics research. Journal of Economic Literature 56 (3), 920-80.

Clemens, M. A. (2015). The meaning of failed replications: A review and proposal. Journal of Economic Surveys 31 (1), 326-342.

Cochran, W. G. and D. B. Rubin (1973). Controlling bias in observational studies: A review. Sankhya: The Indian Journal of Statistics, Series A 35 (4), 417-446.

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ, USA: Lawrence Erlbaum Associates.

Cooper, D. J. and H. Fang (2008). Understanding overbidding in second price auctions: An experimental study. The Economic Journal 118 (532), 1572-1595.

Corgnet, B., R. Hernan-Gonzalez, P. Kujal, and D. Porter (2014). The effect of earned versus house money on price bubble formation in experimental asset markets. Review of Finance 19 (4), 1455-1488.

Corrigan, J. R., A. C. Drichoutis, J. L. Lusk, J. R.M. Nayga, and M. C. Rousu (2012). Repeated rounds with price feedback in experimental auction valuation: An adversarial collaboration. American Journal of Agricultural Economics 94 (1), 97-115.

Cragg, J. G. (1971). Some statistical models for limited dependent variables with application to the demand for durable goods. Econometrica 39 (5), 829-844.

Croson, R. (2005). The method of experimental economics. International Negotiation 10 (1), 131-148.

Davis, L. R., B. P. Joyce, and M. R. Roelofs (2010). My money or yours: house money payment effects. Experimental Economics 13 (2), 189-205.

Deaton, A. and N. Cartwright (2016). Understanding and misunderstanding randomized controlled trials. National Bureau of Economic Research Working Paper No. 22595.

Debnam, J. (2017). Selection effects and heterogeneous demand responses to the Berkeley soda tax vote. American Journal of Agricultural Economics 99 (5), 1172-1187.

Delgado, M. R., A. Schotter, E. Y. Ozbay, and E. A. Phelps (2008). Understanding overbidding: Using the neural circuitry of reward to design economic auctions. Science 321 (5897), 1849-1852.

Demont, M., P. Rutsaert, M. Ndour, W. Verbeke, P. A. Seck, and E. Tollens (2013). Experimental auctions, collective induction and choice shift: willingness-to-pay for rice quality in Senegal. European Review of Agricultural Economics 40 (2), 261-286.

Demont, M., E. Zossou, P. Rutsaert, M. Ndour, P. Van Mele, and W. Verbeke (2012). Consumer valuation of improved rice parboiling technologies in benin. Food Quality and Preference 23 (1), 63-70.

Dewald, W. G., J. G. Thursby, and R. G. Anderson (1986). Replication in empirical economics: The Journal of Money, Credit and Banking project. The American Economic Review 76 (4), 587-603.

Diggle, P. J., P. Heagerty, K.-Y. Liang, and S. L. Zeger (2002). Analysis of Longitudinal Data (2nd ed.). New York, USA: Oxford University Press Inc.

Dillaway, R., K. D. Messer, J. C. Bernard, and H. M. Kaiser (2011). Do consumer responses to media food safety information last? Applied Economic Perspectives and Policy 33 (3), 363-383.

Dreyfyss, E. (2018, August 17). A bot panic hits Amazon's Mechanical Turk. Wired. Available at https://perma.cc/42U2-AQ58. Last accessed on September 9, 2018.

Drichoutis, A. C., S. Klonaris, and G. S. Papoutsi (2017). Do good things come in small packages? Bottle size effects on willingness to pay for pomegranate wine and grape wine. Journal of Wine Economics 12 (1), 84-104.

Drichoutis, A. C. and J. L. Lusk (2014). Judging statistical models of individual decision making under risk using in- and out-of-sample criteria. PLoS ONE 9 (7), e102269.

Drichoutis, A. C. and J. L. Lusk (2016). What can multiple price lists really tell us about risk preferences? Journal of Risk and Uncertainty 53 (2), 89-106.

Drichoutis, A. C., J. L. Lusk, and R. M. Nayga (2015). The veil of experimental currency units in second price auctions. Journal of the Economic Science Association 1 (2), 182-196.

Drichoutis, A. C., A. Vassilopoulos, and J. L. Lusk (2014). Consumers' willingness to pay for agricultural products certified to ensure fair working conditions. Report to the John S. Latsis Public Benefit Foundation. Available at https://perma.cc/LFP7-XBJM. Last accessed on December 8, 2017.

Drichoutis, A. C., A. Vassilopoulos, J. L. Lusk, and J. R. M. Nayga (2017). Consumer preferences for fair labour certification. European Review of Agricultural Economics 44 (3), 455-474.

Duval, S. and R. Tweedie (2000). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 56 (2), 455-463.

Dyer, D. and J. H. Kagel (1996). Bidding in common value auctions: How the commercial construction industry corrects for the winner's curse. Management Science 42 (10), 1463-1475.

Egger, M., G. D. Smith, M. Schneider, and C. Minder (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ 315 (7109), 629-634.

Ellison, B., J. L. Lusk, and D. Davis (2014). The impact of restaurant calorie labels on food choice: Results from a field experiment. Economic Inquiry 52 (2), 666-681.

Erdem, T. (1996). A dynamic analysis of market structure based on panel data. Marketing Science 15 (4), 359-378.

Fafchamps, M. and J. Labonne (2016). Using split samples to improve inference about causal effects. National Bureau of Economic Research Working Paper Series No. 21842.

Fanelli, D. and J. P. A. Ioannidis (2013). US studies may overestimate effect sizes in softer research. Proceedings of the National Academy of Sciences.

Feuz, D. M., W. J. Umberger, C. R. Calkins, and B. Sitz (2004). U.S. consumers' willingness to pay for flavor and tenderness in steaks as determined with an experimental auction. Journal of Agricultural and Resource Economics 29 (3), 501-516.

Fidler, F., N. Thomason, G. Cumming, S. Finch, and J. Leeman (2004). Editors can lead researchers to confidence intervals, but can't make them think:statistical reform lessons from medicine. Psychological Science 15 (2), 119-126.

Fisher, R. A. (1925). Statistical Methods for Research Workers. Edinburgh: Oliver and Boyd.

Flynn, N., C. Kah, and R. Kerschbamer (2016). Vickrey auction vs bdm: difference in bidding behaviour and the impact of other-regarding motives. Journal of the Economic Science Association 2 (2), 101-108.

Fox, J. A., J. F. Shogren, D. J. Hayes, and J. B. Kliebenstein (1998). CVM-X: Calibrating contingent values with experimental auction markets. American Journal of Agricultural Economics 80 (3), 455-465.

Gelman, A. (2018). Don't calculate post-hoc power using observed estimate of effect size. Annals of Surgery In Press.

Gelman, A. and J. Carlin (2014). Beyond power calculations:assessing type s (sign) and type m (magnitude) errors. Perspectives on Psychological Science 9 (6), 641-651.

Georganas, S., D. Levin, and P. McGee (2017). Optimistic irrationality and overbidding in private value auctions. Experimental Economics 20 (4), 772-792.

Gigerenzer, G., S. Krauss, and O. Vitouch (2004). The Null Ritual: What You Always Wanted to Know About Significance Testing but Were Afraid to Ask, pp. 391-408. Thousand Oaks, CA: SAGE Publications, Inc.

Gill, J. (2018). Comments from the new editor. Political Analysis 26 (1), 1-2.

Gneezy, A. (2016). Field experimentation in marketing research. Journal of Marketing Research 54 (1), 140-143.

Gracia, A., M. L. Loureiro, and J. R. M. Nayga (2011). Are valuations from nonhypothetical choice experiments different from those of experimental auctions? American Journal of Agricultural Economics 93 (5), 1358-1373.

Grebitus, C., J. L. Lusk, and R. M. Nayga (2013). Explaining differences in real and hypothetical experimental auctions and choice experiments with personality. Journal of Economic Psychology 36 (Supplement C), 11-26.

Grether, D. M., C. R. Plott, D. B. Rowe, M. Sereno, and J. M. Allman (2007). Mental processes and strategic equilibration: An fmri study of selling strategies in second price auctions. Experimental Economics 10 (2), 105-122.

Ham, J. C. and J. H. Kagel (2006). Gender effects in private value auctions. Economics Letters 92 (3), 375-382.

Hankins, M. (2013, April 21). Still not significant. Probable Error. Available at http://perma.cc/Z6B9-PHCV. Last accessed on August 6, 2018.

Harrison, G. W. (1989). Theory and misbehavior of First-price auctions. The American Economic Review 79 (4), 749-762.

Harrison, G. W. (1992). Theory and misbehavior of First-price auctions: Reply. The American Economic Review 82 (5), 1426-1443.

Harrison, G. W. (2006). Experimental evidence on alternative environmental valuation methods. Environmental & Resource Economics 36, 125-162.

Harrison, G. W., E. Johnson, M. M. McInnes, and E. E. Rutstrom (2005). Temporal stability of estimates of risk aversion. Applied Financial Economics Letters 1 (1), 31-35.

Harrison, G. W. and J. A. List (2004). Field experiments. Journal of Economic Literature 42 (4), 1009-1055.

Hedges, L. V. (1981). Distribution theory for glass's estimator of effect size and related estimators. Journal of Educational Statistics 6 (2), 107-128.

Ho, D. E., K. Imai, G. King, and E. A. Stuart (2007). Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis 15 (3), 199-236.

Hoenig, J. M. and D. M. Heisey (2001). The abuse of power. The American Statistician 55 (1), 19-24.

Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association 81 (396), 945-960.

Holmquist, C., J. McCluskey, and C. Ross (2012). Consumer preferences and willingness to pay for oak attributes in washington chardonnays. American Journal of Agricultural Economics 94 (2), 556-561.

Horowitz, J. K. (2006). The becker-degroot-marschak mechanism is not necessarily incentive compatible, even for non-random goods. Economics Letters 93 (1), 6-11.

Imai, K., G. King, and E. A. Stuart (2008). Misunderstandings between experimentalists and observationalists about causal inference. Journal of the Royal Statistical Society: Series A (Statistics in Society) 171 (2), 481-502.

Imbens, G. W. and D. B. Rubin (2016). Causal Inference for Statistics, Social, and Biomedical Sciences, An introduction. Cambridge and New York: Cambridge University Press.

Imbens, G. W. and J. M. Wooldridge (2009). Recent developments in the econometrics of program evaluation. Journal of Economic Literature 47 (1), 5-86.

Ioannidis, J. P. A. and T. A. Trikalinos (2007). An exploratory test for an excess of significant Findings. Clinical Trials 4 (3), 245-253.

Jacquemet, N., R.-V. Joule, S. Luchini, and J. F. Shogren (2009). Earned wealth, engaged bidders? Evidence from a second-price auction. Economics Letters 105 (1), 36-38.

Johnson, D. and J. Ryan (2018). Amazon mechanical turk workers can provide consistent and economically meaningful data. Munich Personal RePEc Archive No. 88450.

Johnson, E. J., M. Steffel, and D. G. Goldstein (2005). Making better decisions: From measuring to constructing preferences. Health Psychology 24 (4), S17-S22.

Josephson, A. and J. D. Michler (2018). Viewpoint: Beasts of the field? Ethics in agricultural and applied economics. Food Policy 79, 1-11.

Just, D. R. (2017). The behavioral welfare paradox: Practical, ethical and welfare implications of nudging. Agricultural and Resource Economics Review 46 (1), 1-20.

Just, D. R. and A. S. Hanks (2015). The hidden cost of regulation: Emotional responses to command and control. American Journal of Agricultural Economics 97 (5), 1385-1399.

Kagel, J. H., R. M. Harstad, and D. Levin (1987). Information impact and allocation rules in auctions with affiliated private values: A laboratory study. Econometrica 55 (6), 1275-1304.

Kagel, J. H. and D. Levin (1993). Independent private value auctions: Bidder behaviour in first-, second- and third-price auctions with varying numbers of bidders. The Economic Journal 103 (419), 868-879.

Kaiser, H. F. (1960). Directional statistical decisions. Psychological Review 67 (3), 160-167.

Kang, M. and C. Camerer (2013). fmri evidence of a hot-cold empathy gap in hypothetical and real aversive choices. Frontiers in Neuroscience 7 (104).

Karni, E. and Z. Safra (1987). "Preference reversal" and the observability of preferences by experimental methods. Econometrica 55 (3), 675-685.

Katkar, R. and D. H. Reiley (2007). Public versus secret reserve prices in ebay auctions: Results from a pokemon field experiment. The B.E. Journal of Economic Analysis & Policy 5 (2).

Kechagia, V. and A. C. Drichoutis (2017). The effect of olfactory sensory cues on willingness to pay and choice under risk. Journal of Behavioral and Experimental Economics 70, 33-46.

Kennedy, P. E. (2002). Sinning in the basement: What are the rules? The ten commandments of applied econometrics. Journal of Economic Surveys 16 (4), 569-589.

Kenny, D. A. (1987). Chapter 13: The two-group design, pp. 203-223. Little, Brown.

Keser, C., K.-M. Ehrhart, and S. K. Berninghaus (1998). Coordination and local interaction: experimental evidence. Economics Letters 58 (3), 269-275.

Kessler, J. B. and S. Meier (2014). Learning from (failed) replications: Cognitive load manipulations and charitable giving. Journal of Economic Behavior & Organization 102 (0),10-13.

Kimball, A. W. (1957). Errors of the third kind in statistical consulting. Journal of the American Statistical Association 52 (278), 133-142.

Klain, T. J., J. L. Lusk, G. T. Tonsor, and T. C. Schroeder (2014). An experimental approach to valuing information. Agricultural Economics 45 (5), 635-648.

Kline, R. B. (2013). Beyond Significance Testing: Statistics Reform in the Behavioral Sciences. American Psychological Association.

Kovalsky, K. L. and J. L. Lusk (2013). Do consumers really know how much they are willing to pay? Journal of Consumer Affairs 47 (1), 98-127.

Kupper, L. L. and K. B. Hafner (1989). How appropriate are popular sample size formulas? The American Statistician 43 (2), 101-105.

Lakens, D., F. G. Adolfi C. J. Albers, F. Anvari, M. A. J. Apps, S. E. Argamon, T. Baguley, R. B. Becker, S. D. Benning, D. E. Bradford, E. M. Buchanan, A. R. Caldwell, B. Van Calster, R. Carlsson, S.-C. Chen, B. Chung, L. J. Colling, G. S. Collins, Z. Crook, E. S. Cross, S. Daniels, H. Danielsson, L. DeBruine, D. J. Dunleavy, B. D. Earp, M. I. Feist, J. D. Ferrell, J. G. Field, N. W. Fox, A. Friesen, C. Gomes, M. Gonzalez-Marquez, J. A. Grange, A. P. Grieve, R. Guggenberger, J. Grist, A.-L. van Harmelen, F. Hasselman, K. D. Hochard, M. R. Hoffarth, N. P. Holmes, M. Ingre, P. M. Isager, H. K. Isotalus, C. Johansson, K. Juszczyk, D. A. Kenny, A. A. Khalil, B. Konat, J. Lao, E. G. Larsen, G. M. A. Lodder, J. Lukavsky, C. R. Madan, D. Manheim, S. R. Martin, A. E. Martin, D. G. Mayo, R. J. McCarthy, K. McConway, C. McFarland, A. Q. X. Nio, G. Nilsonne, C. L. de Oliveira, J.-J. O. de Xivry, S. Parsons, G. Pfuhl, K. A. Quinn, J. J. Sakon, S. A. Saribay, I. K. Schneider, M. Selvaraju, Z. Sjoerds, S. G. Smith, T. Smits, J. R. Spies, V. Sreekumar, C. N. Steltenpohl, N. Stenhouse, W. Swiatkowski, M. A. Vadillo, M. A. L. M. Van Assen, M. N. Williams, S. E. Williams, D. R. Williams, T. Yarkoni, I. Ziano, and R. A. Zwaan (2018). Justify your alpha. Nature Human Behaviour 2 (3), 168-171.

Lane, D. M. and W. P. Dunlap (1978). Estimating effect size: Bias resulting from the significance criterion in editorial decisions. British Journal of Mathematical and Statistical Psychology 31 (2), 107-112.

Lee, J. Y. and J. A. Fox (2015). Bidding behavior in experimental auctions with positive and negative values. Economics Letters 136, 151-153.

Lee, J. Y., R. M. J. Nayga, C. Deck, and A. C. Drichoutis (2017). Cognitive ability and bidding behavior in second price auctions: An experimental study. Munich Personal RePEc Archive No. 81495.

Lehner, R., J. H. Balsters, A. Herger, T. A. Hare, and N. Wenderoth (2017). Monetary, food, and social rewards induce similar pavlovian-to-instrumental transfer effects. Frontiers in Behavioral Neuroscience 10 (247).

Lerner, J., D. Small, and G. Loewenstein (2004). Heart strings and purse strings: Carryover effects of emotions on economic decisions. Psychological Science 15 (5), 337-341.

Lewis, K. E., C. Grebitus, and R. M. Nayga (2016a). The impact of brand and attention on consumers' willingness to pay: Evidence from an eye tracking experiment. Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie 64 (4), 753-777.

Lewis, K. E., C. Grebitus, and R. M. Nayga (2016b). The importance of taste in experimental auctions: consumers' valuation of calorie and sweetener labeling of soft drinks. Agricultural Economics 47 (1), 47-57.

Li, S. (2017). Obviously strategy-proof mechanisms. American Economic Review 107 (11), 3257-3287.

Lichtenstein, S. and P. Slovic (2006). The construction of preference. Cambridge, UK: Cambridge University Press.

Linder, N. S., G. Uhl, K. Fliessbach, P. Trautner, C. E. Elger, and B. Weber (2010). Organic labeling influences food valuation and choice. NeuroImage 53 (1), 215-220.

List, J. A. (2003). Does market experience eliminate market anomalies?. The Quarterly Journal of Economics 118 (1), 41-71.

List, J. A. (2004). Neoclassical theory versus prospect theory: Evidence from the marketplace. Econometrica 72 (2), 615-625.

List, J. A. and D. Lucking-Reiley (2000). Demand reduction in multiunit auctions: Evidence from a sportscard field experiment. The American Economic Review 90 (4), 961-972.

List, J. A., S. Sadoff, and M. Wagner (2011). So you want to run an experiment, now what? some simple rules of thumb for optimal experimental design. Experimental Economics 14 (4), 439.

List, J. A. and J. F. Shogren (1998). Calibration of the difference between actual and hypothetical valuations in a field experiment. Journal of Economic Behavior & Organization 37 (2), 193-205.

Liu, H. and T. Wu (2005). Sample size calculation and power analysis of time-averaged difference. Journal of Modern Applied Statistical Methods 4 (2), 434-445.

Lucking-Reiley, D., D. Bryan, N. Prasad, and D. Reeves (2007). Pennies from ebay: The determinants of price in online auctions*. The Journal of Industrial Economics 55 (2), 223-233.

Lusk, J. (2010). Experimental auction markets for studying consumer preferences. In S. R. Jaeger and H. MacFie (Eds.), Consumer-Driven Innovation in Food and Personal Care Products, Woodhead Publishing Series in Food Science, Technology and Nutrition, pp. 332 - 357. Woodhead Publishing.

Lusk, J. L. (2014). Are you smart enough to know what to eat? a critique of behavioural economics as justification for regulation. European Review of Agricultural Economics 41 (3), 355-373.

Lusk, J. L. (2017). Consumer research with big data: Applications from the food demand survey (foods). American Journal of Agricultural Economics 99 (2), 303-320.

Lusk, J. L., C. Alexander, and M. C. Rousu (2007). Designing experimental auctions for marketing research: The effect of values, distributions, and mechanisms on incentives for truthful bidding. Review of Marketing Science 5 (1).

Lusk, J. L., M. S. Daniel, D. R. Mark, and C. L. Lusk (2001). Alternative calibration and auction institutions for predicting consumer willingness to pay for nongenetically modified corn chips. Journal of Agricultural and Resource Economics 26 (1), 40-57.

Lusk, J. L. and J. A. Fox (2002). Consumer demand for mandatory labeling of beef from cattle administered growth hormones or fed genetically modified corn. Journal of Agricultural and Applied Economics 34 (1), 27-38.

Lusk, J. L. and J. A. Fox (2003). Value elicitation in retail and laboratory environments. Economics Letters 79 (1), 27-34.

Lusk, J. L., S. Marette, and F. B. Norwood (2014). The paternalist meets his match. Applied Economic Perspectives and Policy 36 (1), 61-108.

Lusk, J. L., F. B. Norwood, and J. R. Pruitt (2006). Consumer demand for a ban on antibiotic drug use in pork production. American Journal of Agricultural Economics 88 (4), 1015-1033.

Lusk, J. L., J. R. Pruitt, and B. Norwood (2006). External validity of a framed field experiment. Economics Letters 93 (2), 285-290.

Lusk, J. L. and T. C. Schroeder (2004). Are choice experiments incentive compatible? a test with quality differentiated beef steaks. American Journal of Agricultural Economics 86 (2), 467-482.

Lusk, J. L. and T. C. Schroeder (2006). Auction bids and shopping choices. Advances in Economic Analysis & Policy 6 (1).

Lusk, J. L. and J. F. Shogren (2007). Experimental auctions, Methods and applications in economic and marketing research. Cambridge, UK: Cambridge University Press.

Malone, T. and J. L. Lusk (2018). Releasing the trap: A method to reduce inattention bias in survey data with application to u.s. beer taxes. Economic Inquiry (forthcoming).

Maniadis, Z., F. Tufano, and J. A. List (2014). One swallow doesn't make a summer: New evidence on anchoring effects. American Economic Review 104 (1), 277-90.

Mazar, N., B. Koszegi, and D. Ariely (2013). True context-dependent preferences? the causes of market-dependent valuations. Journal of Behavioral Decision Making.

McCullough, B., K. A. McGeary, and T. D. Harrison (2008). Do economics journal archives promote replicable research? Canadian Journal of Economics/Revue Canadienne d'Economique 41 (4), 1406-1420.

McCullough, B. D., K. A. McGeary, and T. D. Harrison (2006). Lessons from the JMCB archive. Journal of Money, Credit and Banking 38 (4), 1093-1107.

Melton, B. E., W. E. Huffman, J. F. Shogren, and J. A. Fox (1996). Consumer preferences for fresh food items with multiple quality attributes: Evidence from an experimental auction of pork chops. American Journal of Agricultural Economics 78 (4), 916-923.

Miguel, E., C. Camerer, K. Casey, J. Cohen, K. M. Esterling, A. Gerber, R. Glennerster, D. P. Green, M. Humphreys, G. Imbens, D. Laitin, T. Madon, L. Nelson, B. A. Nosek, M. Petersen, R. Sedlmayr, J. P. Simmons, U. Simonsohn, and M. Van der Laan (2014). Promoting transparency in social science research. Science 343 (6166), 30-31.

Moher, D., S. Hopewell, K. F. Schulz, V. Montori, P. C. Gotzsche, P. J. Devereaux, D. Elbourne, M. Egger, and D. G. Altman (2010). CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 340.

Morgan, J., K. Steiglitz, and G. Reis (2003). The spite motive and equilibrium behavior in auctions. Contributions in Economic Analysis & Policy 2 (1).

Muller, L., L. Anne, L. J. L., and R. Bernard (2017). Distributional impacts of fat taxes and thin subsidies. The Economic Journal 127 (604), 2066-2092.

Norwood, B. F., M. C. Roberts, and J. L. Lusk (2004). Ranking crop yield models using out-of sample likelihood functions. American Journal of Agricultural Economics 86 (4), 1032-1043.

Nosek, B. A., G. Alter, G. C. Banks, D. Borsboom, S. D. Bowman, S. J. Breckler, S. Buck, C. D. Chambers, G. Chin, G. Christensen, M. Contestabile, A. Dafoe, E. Eich, J. Freese, R. Glennerster, D. Goroff, D. P. Green, B. Hesse, M. Humphreys, J. Ishiyama, D. Karlan, A. Kraut, A. Lupia, P. Mabry, T. Madon, N. Malhotra, E. Mayo-Wilson, M. McNutt, E. Miguel, E. L. Paluck, U. Simonsohn, C. Soderberg, B. A. Spellman, J. Turitto, G. VandenBos, S. Vazire, E. J. Wagenmakers, R. Wilson, and T. Yarkoni (2015). Promoting an open research culture. Science 348 (6242), 1422-1425.

Nuzzo, R. (2014). Scientific method: Statistical errors - p values, the `gold standard' of statistical validity, are not as reliable as many scientists assume. Nature 506, 150-152.

Olken, B. A. (2015). Promises and perils of pre-analysis plans. Journal of Economic Perspectives 29 (3), 61-80.

Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science 349 (6251).

Orquin, J. L. and S. Mueller Loose (2013). Attention and choice: A review on eye movements in decision making. Acta Psychologica 144 (1), 190-206.

Orwin, R. G. (1983). A fail-safe N for effect size in meta-analysis. Journal of Educational Statistics 8 (2), 157-159.

Parkhurst, G. M., J. F. Shogren, and D. L. Dickinson (2004). Negative values in vickrey auctions. American Journal of Agricultural Economics 86 (1), 222-235.

Payne, J. W., J. R. Bettman, and D. A. Schkade (1999). Measuring constructed preferences: Towards a building code. Journal of Risk and Uncertainty 19 (1), 243-270.

Pearson, M. and B. C. Schipper (2013). Menstrual cycle and competitive bidding. Games and Economic Behavior 78 (forthcoming), 1-20.

Plassmann, H., J. O'Doherty, and A. Rangel (2007). Orbitofrontal cortex encodes willingness to pay in everyday economic transactions. The Journal of Neuroscience 27 (37), 9984-9988.

Platter, W. J., J. D. Tatum, K. E. Belk, S. R. Koontz, P. L. Chapman, and G. C. Smith (2005). Effects of marbling and shear force on consumers' willingness to pay for beef strip loin steaks. Journal of Animal Science 83 (4), 890-899.

Pritschet, L., D. Powell, and Z. Horne (2016). Marginally significant effects as evidence for hypotheses:changing attitudes over four decades. Psychological Science 27 (7), 1036-1042.

Richards, T. J., S. F. Hamilton, and W. J. Allender (2014). Social networks and new product choice. American Journal of Agricultural Economics 96 (2), 489-516.

Rihn, A. L. and C. Yue (2016). Visual attention's in uence on consumers' willingness-to-pay for processed food products. Agribusiness 32 (3), 314-328.

Roe, B. E. and D. R. Just (2009). Internal and external validity in economics research: Tradeoffs between experiments, field experiments, natural experiments, and field data. American Journal of Agricultural Economics 91 (5), 1266-1271.

Roider, A. and P. W. Schmitz (2012). Auctions with anticipated emotions: Overbidding, underbidding, and optimal reserve prices*. The Scandinavian Journal of Economics 114 (3), 808-830.

Rosato, A. and A. A. Tymula (2016). Loss aversion and competition in Vickrey auctions: Money ain't no good. The University of Sydney, Economics Working Paper Series 2016 - 09 .

Rosenbaum, P. R. (2002). Observational studies (2nd ed.). New York: Springer-Verlag.

Rosenboim, M. and T. Shavit (2012). Whose money is it anyway? using prepaid incentives in experimental economics to create a natural environment. Experimental Economics 15 (1), 145-157.

Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin 86 (3), 638-641.

Rowe, D. B. (2001). Bayesian source separation for reference function determination in fmri. Magnetic Resonance in Medicine 46 (2), 374-378.

Roy, R., P. K. Chintagunta, and S. Haldar (1996). A framework for investigating habits, "the hand of the past," and heterogeneity in dynamic brand choice. Marketing Science 15 (3), 280-299.

Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology 66 (5), 688-701.

Rubin, D. B. (1990). [On the application of probability theory to agricultural experiments. Essay on principles. Section 9.] Comment: Neyman (1923) and causal inference in experiments and observational studies. Statistical Science 5 (4), 472-480.

Schipper, B. C. (2015). Sex hormones and competitive bidding. Management Science 61 (2), 249-486.

Selya, A. S., J. S. Rose, L. C. Dierker, D. Hedeker, and R. J. Mermelstein (2012). A practical guide to calculating Cohen's f2, a measure of local effect size, from PROC MIXED. Frontiers in Psychology 3, 111.

Senn, S. (1994). Testing for baseline balance in clinical trials. Statistics in Medicine 13 (17), 1715-1726.

Senn, S. (2013). Seven myths of randomisation in clinical trials. Statistics in Medicine 32 (9), 1439-1450.

Shogren, J. F. (2006). Valuation in the lab. Environmental & Resource Economics 34 (163-172).

Shogren, J. F., J. A. List, and D. J. Hayes (2000). Preference learning in consecutive experimental auctions. American Journal of Agricultural Economics 82 (4), 1016-1021.

Shogren, J. F., M. Margolis, C. Koo, and J. A. List (2001). A random nth-price auction. Journal of Economic Behavior and Organization 46 (4), 409-421.

Simmons, J., L. Nelson, and U. Simonsohn (2012). A 21 word solution. Dialogue: The Official Newsletter of the Society for Personality and Social Psychology 26 (2), 4-7.

Simmons, J. P., L. D. Nelson, and U. Simonsohn (2011). False-positive psychology: Undisclosed exibility in data collection and analysis allows presenting anything as signifficant. Psychological Science 22 (11), 1359-1366.

Simonsohn, U. (2015). Small telescopes: Detectability and the evaluation of replication results. Psychological Science 26 (5), 559-569.

Simonsohn, U., L. D. Nelson, and J. P. Simmons (2014). P-curve: A key to the file-drawer. Journal of Experimental Psychology: General 143 (2), 534-547.

Simonsohn, U., J. P. Simmons, and L. D. Nelson (2013). Anchoring is not a false-positive: Maniadis, tufano, and list's (2014) `failure-to-replicate' is actually entirely consistent with the original. Working paper, Available at http://dx.doi.org/10.2139/ssrn.2351926 .

Slovic, P. (1995). The construction of preference. American Psychologist 50 (5), 364-371.

Sousa, Y. F. D. and A. Munro (2012). Truck, barter and exchange versus the endowment effect: Virtual field experiments in an online game environment. Journal of Economic Psychology 33 (3), 482-493.

Speed, T. P. (1990). Introductory remarks on Neyman (1923). Statistical Science 5 (4), 463-464.

Splawa-Neyman, J., D. M. Dabrowska, and T. P. Speed (1990). On the application of probability theory to agricultural experiments. essay on principles. section 9. Statistical Science 5 (4), 465-472.

Stigler, G. J. and G. S. Becker (1977). De Gustibus Non Est Disputandum. The American Economic Review 67 (2), 76-90.

Toler, S., B. C. Briggeman, J. L. Lusk, and D. C. Adams (2009). Fairness, farmers markets, and local production. American Journal of Agricultural Economics 91 (5), 1272-1278.

Trafimow, D. and M. Marks (2015). Editorial. Basic and Applied Social Psychology 37 (1), 1-2.

Tyson-Carr, J., K. Kokmotou, V. Soto, S. Cook, N. Fallon, T. Giesbrecht, and A. Stancak (2018). Neural correlates of economic value and valuation context: an event-related potential study. Journal of Neurophysiology 119 (5), 1924-1933.

Umberger, W. J. and D. M. Feuz (2004). The usefulness of experimental auctions in determining consumers' willingness-to-pay for quality-differentiated products. Applied Economic Perspectives and Policy 26 (2), 170-185.

Umberger, W. J., D. M. Feuz, C. R. Calkins, and K. Killinger-Mann (2002). U.s. consumer preference and willingness-to-pay for domestic corn-fed beef versus international grass-fed beef measured through an experimental auction. Agribusiness 18 (4), 491-504.

Urbancic, M. (2011). Testing distributional dependence in the becker-degroot-marschak mechanism. Working paper, UC Berkeley.

Vassilopoulos, A., A. C. Drichoutis, and R. M. Nayga Jr (2018). Loss aversion, expectations and anchoring in the bdm mechanism. Munich Personal RePEc Archive No. 85635.

Vecchio, R. and M. Borrello (2018). Measuring food preferences through experimental auctions: A review. Food Research International.

Veling, H., Z. Chen, M. C. Tombrock, I. A. M. Verpaalen, L. I. Schmitz, A. Dijksterhuis, and R. W. Holland (2017). Training impulsive choices for healthy and sustainable food. Journal of Experimental Psychology: Applied 23 (2), 204-215.

Wasserstein, R. L. and N. A. Lazar (2016). The asa's statement on p-values: Context, process, and purpose. The American Statistician 70 (2), 129-133.

Wedel, M. and R. Pieters (2000). Eye fixations on advertisements and memory for brands: A model and findings. Marketing Science 19 (4), 297-312.

West, S. G. and F. Thoemmes (2010). Campbell's and Rubin's perspectives on causal inference. Psychological Methods 15 (1), 18-37.

Zhang, L. and A. Ortmann (2013). Exploring the meaning of significance in experimental economics. UNSW Australian School of Business Research Paper No. 2013-32.

Zhang, Y. Y., R. M. Nayga, and D. P. T. Depositario (2017). Learning and the possibility of losing own money reduce overbidding: Delayed payment in experimental auctions. Social Scence Research Network Available at http://dx.doi.org/10.2139/ssrn.2636040.

Zizzo, D. J. (2010). Experimenter demand effects in economic experiments. Experimental Economics 13 (1), 75-98.