Relative Performance Feedback and the Effects of Being Above Average - Field Experiment and Replication

Brade, Raphael and Himmler, Oliver and Jäckle, Robert (2022): Relative Performance Feedback and the Effects of Being Above Average - Field Experiment and Replication.

Abstract

In a randomized field experiment, we give first-year students in higher education feedback on their relative performance and show that the type of feedback matters, as feedback increases performance only if it informs the student that they placed above average in the past. We reproduce the results in a replication experiment and investigate mechanisms: The effects are not driven by above-average students reacting particularly well to feedback due to individual characteristics; rather, the information about being above average makes feedback effective. We present evidence that individuals focus on good news to adjust their beliefs, and that feedback can offset disadvantages faced by individuals who are held back by their own underestimation of relative abilities. Once beliefs between controls and the treated converge, repeated treatment does not add to the effects.

• Item Type: MPRA Paper
• Language: English
• Keywords: Relative Performance Feedback; Randomized Field Experiment; Selective Information Processing; Higher Education
• Subjects: C - Mathematical and Quantitative Methods > C9 - Design of Experiments > C93 - Field Experiments; I - Health, Education, and Welfare > I2 - Education and Research Institutions > I23 - Higher Education ; Research Institutions

References

Allcott, Hunt. 2011. “Social Norms and Energy Conservation.” Journal of Public Economics, 95(9–10): 1082–1095.

Allcott, Hunt, and Todd Rogers. 2014. “The Short-Run and Long-Run Effects of Behavioral Interventions: Experimental Evidence from Energy Conservation.” American Economic Review, 104(10): 3003–3037.

Altmann, Steffen, Andreas Grunewald, and Jonas Radbruch. in press. “Interventions and Cognitive Spillovers.” Review of Economic Studies.

Ashraf, Anik. 2022. “Performance Ranks, Conformity, and Cooperation: Evidence from a Sweater Factory.” CESifo Working Paper No. 9591.

Attewell, Paul, and David Monaghan. 2016. “How Many Credits Should an Undergraduate Take?” Research in Higher Education, 57(6): 682–713.

Attewell, Paul, Scott Heil, and Liza Reisel. 2012. “What is Academic Momentum? And Does it Matter?” Educational Evaluation and Policy Analysis, 34(1): 27–44.

Azmat, Ghazala, and Nagore Iriberri. 2010. “The Importance of Relative Performance Feedback Information: Evidence from a Natural Experiment using High School Students.” Journal of Public Economics, 94(7): 435–452.

Azmat, Ghazala, and Nagore Iriberri. 2016. “The Provision of Relative Performance Feedback: An Analysis of Performance and Satisfaction.” Journal of Economics & Management Strategy, 25(1): 77–110.

Azmat, Ghazala, Manuel Bagues, Antonio Cabrales, and Nagore Iriberri. 2019. “What You Don’t Know. . . Can’t Hurt You? A Natural Field Experiment on Relative Performance Feedback in Higher Education.” Management Science, 65(8): 3449–3947.

Bandiera, Oriana, Iwan Barankay, and Imran Rasul. 2013. “Team Incentives: Evidence from a Firm Level Experiment.” Journal of the European Economic Association, 11(5): 1079–1114.

Bandura, Albert. 1991. “Social Cognitive Theory of Self-Regulation.” Organizational Behavior and Human Decision Processes, 50(2): 248 – 287.

Barankay, Iwan. 2012. “Rank Incentives: Evidence from a Randomized Workplace Experiment.” mimeo.

Bénabou, Roland, and Jean Tirole. 2002. “Self-Confidence and Personal Motivation.” Quarterly Journal of Economics, 117(3): 871–915.

Blanes i Vidal, Jordi, and Mareike Nossol. 2011. “Tournaments Without Prizes: Evidence from Personnel Records.” Management Science, 57(10): 1721–1736.

Bound, John, Michael F. Lovenheim, and Sarah Turner. 2012. “Increasing Time to Baccalaureate Degree in the United States.” Education Finance and Policy, 7(4): 375–424.

Bruhn, Miriam, and David McKenzie. 2009. “In Pursuit of Balance: Randomization in Practice in Development Field Experiments.” American Economic Journal: Applied Economics, 1(4): 200–232.

Bryan, Christopher J., Elizabeth Tipton, and David S. Yeager. 2021. “Behavioural Science Is Unlikely to Change the World Without a Heterogeneity Revolution.” Nature Human Behaviour, 5(8): 1–10.

Bundesamt, Statistisches. 2020. “Bildung und Kultur: Studierende an Hochschulen Wintersemester 2019/2020.”

Cabrera, José María, and Alejandro Cid. 2017. “Gender Differences to Relative Performance Feedback: A Field Experiment in Education.” mimeo.

Camerer, Colin F., Anna Dreber, Eskil Forsell, Teck-Hua Ho, Jürgen Huber, Magnus Johannesson, Michael Kirchler, Johan Almenberg, Adam Altmejd, Taizan Chan, Emma Heikensten, Felix Holzmeister, Taisuke Imai, Siri Isaksson, Gideon Nave, Thomas Pfeiffer, Michael Razen, and Hang Wu. 2016. “Evaluating Replicability of Laboratory Experiments in Economics.” Science, 351(6280): 1433–1436.

Celik Katreniak, Dagmara. 2018. “Dark Side of Incentives: Evidence From a Randomized Control Trial in Uganda.” mimeo.

Charness, Gary, David Masclet, and Marie Claire Villeval. 2014. “The Dark Side of Competition for Status.” Management Science, 60(1): 38–55.

Cialdini, Robert B. 2003. “Crafting Normative Messages to Protect the Environment.” Current Directions in Psychological Science, 12(4): 105–109.

Cialdini, Robert B. 2011. “The Focus Theory of Normative Conduct.” In Handbook of Theories of Social Psychology. Vol. 2, , ed. Paul A. M. Van Lage, Arie W. Kruglanski and E. Tory Higgins, 295–312. Sage Thousand Oaks, CA.

Cialdini, Robert B., Linda J. Demaine, Brad J. Sagarin, Daniel W. Barrett, Kelton Rhoads, and Patricia L. Winter. 2006. “Managing Social Norms for Persuasive Impact.” Social Influence, 1(1): 3–15.

Cialdini, Robert B., Raymond R. Reno, and Carl A. Kallgren. 1990. “A Focus Theory of Normative Conduct: Recycling the Concept of Norms to Reduce Littering in Public Places.” Journal of Personality and Social Psychology, 58(6): 1015–1026.

Clark, Damon, David Gill, Victoria Prowse, and Mark Rush. 2020. “Using Goals to Motivate College Students: Theory and Evidence From Field Experiments.” Review of Economics and Statistics,102(4): 648–663.

Compte, Olivier, and Andrew Postlewaite. 2004. “Confidence-Enhanced Performance.” American Economic Review, 94(5): 1536–1557.

Corcoran, Katja, Jan Crusius, and Thomas Mussweiler. 2011. “Social Comparison: Motives, Standards, and Mechanisms.” In Theories in Social Psychology. , ed. Derek Chadee, 119–139. Wiley-Blackwell.

Costa, Dora L., and Matthew E. Kahn. 2013. “Energy Conservation “Nudges” and Environmentalist Ideology: Evidence from a Randomized Residential Electricity Field Experiment.” Journal of the European Economic Association, 11(3): 680–702.

Czibor, Eszter, David Jimenez-Gomez, and John A. List. 2019. “The Dozen Things Experimental Economists Should Do (More of).” Southern Economic Journal, 86(2): 371–432.

Danzer, Natalia, and Victor Lavy. 2018. “Paid Parental Leave and Children’s Schooling Outcomes.” Economic Journal, 128(608): 81–117.

Delfgaauw, Josse, Robert Dur, Joeri Sol, and Willem Verbeke. 2013. “Tournament Incentives in the Field: Gender Differences in the Workplace.” Journal of Labor Economics, 31(2): 305–326.

DellaVigna, Stefano, and Elizabeth Linos. 2022. “RCTs to Scale: Comprehensive Evidence from Two Nudge Units.” Econometrica, 90(1): 81–116.

Dobrescu, Loretti Isabella, Marco Faravelli, Rigissa Megalokonomou, and Alberto Motta. 2021. “Relative Performance Feedback in Education: Evidence from a Randomised Controlled Trial.” Economic Journal, 131(640): 3145–3181.

Doyle, William R. 2011. “Effect of Increased Academic Momentum on Transfer Rates: An Application of the Generalized Propensity Score.” Economics of Education Review, 30(1): 191–200.

Dustmann, Christian, and Uta Schönberg. 2012. “Expansions in Maternity Leave Coverage and Children’s Long-Term Outcomes.” American Economic Journal: Applied Economics, 4(3): 190–224.

Duvendack, Maren, Richard Palmer-Jones, and W. Robert Reed. 2017. “What Is Meant by "Replication" and Why Does It Encounter Resistance in Economics?” American Economic Review, 107(5): 46–51.

Eil, David, and Justin M. Rao. 2011. “The Good News-Bad News Effect: Asymmetric Processing of Objective Information about Yourself.” American Economic Journal: Microeconomics, 3(2): 114–138.

Ellis, Albert, and William J. Knaus. 1977. Overcoming Procrastination. New York:Signet Books.

Elsner, Benjamin, and Ingo E. Isphording. 2017. “A Big Fish in a Small Pond: Ability Rank and Human Capital Investment.” Journal of Labor Economics, 35(3): 787–828.

Elsner, Benjamin, Ingo E Isphording, and Ulf Zölitz. 2021. “Achievement Rank Affects Performance and Major Choices in College.” Economic Journal, 131(640): 3182–3206.

Eriksson, Tor, Anders Poulsen, and Marie Claire Villeval. 2009. “Feedback and Incentives: Experimental Evidence.” Labour Economics, 16(6): 679–688.

Ertac, Seda. 2005. “Social Comparisons and Optimal Information Revelation: Theory and Experiments.” Job Market Paper, UCLA.

Festinger, Leon. 1954. “A Theory of Social Comparison Processes.” Human Relations, 7(2): 117–140.

Fischer, Mira, and Valentin Wagner. 2018. “Effects of Timing and Reference Frame of Feedback: Evidence from a Field Experiment.” IZA Discussion Paper No. 11970.

Fryer Jr., Roland G. 2011. “Financial Incentives and Student Achievement: Evidence from Randomized Trials.” Quarterly Journal of Economics, 126(4): 1755–1798.

Gardner, John W. 2020. “Managing Production Yields and Rework through Feedback on Speed, Quality, and Quantity.” Production and Operations Management, 29(9): 2182–2209.

Gill, David, Zdenka Kissová, Jaesun Lee, and Victoria Prowse. 2019. “First-Place Loving and Last-Place Loathing: How Rank in the Distribution of Performance Affects Effort Provision.” Management Science, 65(2): 494–507.

Goldstein, Noah J., Robert B. Cialdini, and Vladas Griskevicius. 2008. “A Room With a Viewpoint: Using Social Norms to Motivate Environmental Conservation in Hotels.” Journal of Consumer Research, 35(3): 472–482.

Goulas, Sofoklis, and Rigissa Megalokonomou. 2021. “Knowing Who You Actually Are: The Effect of Feedback on Short- and Long Term Outcomes.” Journal of Economic Behavior & Organization, 183: 589–615.

Hallsworth, Michael, John A. List, Robert D. Metcalfe, and Ivo Vlaev. 2017. “The Behavioralist as Tax Collector: Using Natural Field Experiments to Enhance Tax Compliance.” Journal of Public Economics, 148: 14–31.

Hamermesh, Daniel S. 2007. “Replication in Economics.” Canadian Journal of Economics/Revue canadienne d’économique, 40(3): 715–733.

Hermes, Henning, Martin Huschens, Franz Rothlauf, and Daniel Schunk. 2021. “Motivating Low-Achievers – Relative Performance Feedback in Primary Schools.” Journal of Economic Behavior & Organization, 187: 45–59.

Hirshleifer, Sarojini R. 2021. “Incentives for Effort or Outputs? A Field Experiment to Improve Student Performance.” CEGA Working Paper Series No. WPS-182.

Hunter, John E. 2001. “The Desperate Need for Replications.” Journal of Consumer Research, 28(1): 149–158.

Ilgen, Daniel, and Cori Davis. 2000. “Bearing Bad News: Reactions to Negative Performance Feedback.” Applied Psychology, 49(3): 550–565.

Ilgen, Daniel R., Cynthia D. Fisher, and M. Susan Taylor. 1979. “Consequences of Individual Feedback on Behavior in Organizations.” Journal of Applied Psychology, 64(4): 349.

Kajitani, Shinya, Keiichi Morimoto, and Shiba Suzuki. 2020. “Information Feedback in Relative Grading: Evidence from a Field Experiment.” PLoS ONE, 15(4): e0231548.

König, Tobias, Sebastian Schweighofer-Kodritsch, and Georg Weizsäcker. 2019. “Beliefs as a Means of Self-Control? Evidence from a Dynamic Student Survey.” WZB Discussion Paper SP II 2019–204.

Köszegi, Botond. 2006. “Ego Utility, Overconfidence, and Task Choice.” Journal of the European Economic Association, 4(4): 673–707.

Kraft, Matthew A. 2020. “Interpreting Effect Sizes of Education Interventions.” Educational Researcher, 49(4): 241–253.

Kuhnen, Camelia M., and Agnieszka Tymula. 2012. “Feedback, Self-Esteem, and Performance in Organizations.” Management Science, 58(1): 94–113.

Lee, David S., and Thomas Lemieux. 2010. “Regression Discontinuity Designs in Economics.” Journal of Economic Literature, 48(2): 281–355.

Leeper, Thomas J. 2019. “Where Have the Respondents Gone? Perhaps We Ate Them All.” Public Opinion Quarterly, 83(S1): 280–288.

Levitt, Steven D., and John A. List. 2009. “Field Experiments in Economics: The Past, the Present, and the Future.” European Economic Review, 53(1): 1–18.

Möbius, Markus M., Muriel Niederle, Paul Niehaus, and Tanya S. Rosenblat. in press. “Managing Self-Confidence: Theory and Experimental Evidence.” Management Science.

Morgan, Kari Lock, and Donald B. Rubin. 2012. “Rerandomization to Improve Covariate Balance in Experiments.” The Annals of Statistics, 40(2): 1263–1282.

Murphy, Richard, and Felix Weinhardt. 2020. “Top of the Class: The Importance of Ordinal Rank.” Review of Economic Studies, 87(6): 2777–2826.

O’Brien, William K. 2002. “Applying the Transtheoretical Model to Academic Procrastination.” Doctoral Dissertation.

OECD. 2019. “Education at a Glance 2019: OECD Indicators.”

Open Science Collaboration. 2015. “Estimating the Reproducibility of Psychological Science.” Science, 349(6251): aac4716.

Oreopoulos, Philip, and Uros Petronijevic. 2019. “The Remarkable Unresponsiveness of College Students to Nudging And What We Can Learn from It.” NBER Working Papers No. 26059.

Oreopoulos, Philip, Richard W. Patterson, Uros Petronijevic, and Nolan G. Pope. 2022. “Low-Touch Attempts to Improve Time Management among Traditional and Online College Students.” Journal of Human Resources, 57(1): 1–43.

Pearce, Jone L., and Lyman W. Porter. 1986. “Employee Responses to Formal Performance Appraisal Feedback.” Journal of Applied Psychology, 71(2): 211.

Schultz, P. Wesley, Jessica M. Nolan, Robert B. Cialdini, Noah J. Goldstein, and Vladas Griskevicius. 2007. “The Constructive, Destructive, and Reconstructive Power of Social Norms.” Psychological Science, 18(5): 429–434.

Slemrod, Joel. 2016. “Tax Compliance and Enforcement: New Research and its Policy Implications.” Ross School of Business Working Paper 1302.

Taylor, Shelley E., Heidi A. Wayment, and Mary Carrillo. 1996. “Social Comparison, Self-Regulation, and Motivation.” In Handbook of Motivation and Cognition, Vol. 3. The Interpersonal Context. , ed. Richard M. Sorrentino and E. Tory Higgins, 3–27. Guilford Press.

Tran, Anh, and Richard Zeckhauser. 2012. “Rank as an Inherent Incentive: Evidence from a Field Experiment.” Journal of Public Economics, 96(9): 645–650.

Trope, Yaacov, and Nira Liberman. 2010. “Construal-Level Theory of Psychological Distance.” Psychological Review, 117(2): 440.

Villeval, Marie Claire. 2020. “Performance Feedback and Peer Effects.” In Handbook of Labor, Human Resources and Population Economics. , ed. Klaus F. Zimmermann. Springer.

Zhang, XiaoLi, Jelle de Vries, René de Koster, and ChenGuang Liu. 2021. “Fast and Faultless? Quantity and Quality Feedback in Order Picking.” Production and Operations Management.