Development of the overconfidence measurement instrument for the economic experiment

Michailova, Julija (2010): Development of the overconfidence measurement instrument for the economic experiment.

Abstract

In this article results of the two experiments, aimed at the development of the instrument (test) that would enable construction of the comprehensive measure of individual overconfidence for the use in economic overconfidence experiments, are presented. Instrument was obtained in a two-stage procedure. In the first experimental phase, a pilot test, consisting of fifty general-knowledge questions of the unknown difficulty, was conducted to divide the items into three difficulty levels: hard, average-difficulty and easy questions. The second phase was aimed at verification of the replicability of results. Statistical tests supported the existence of the hard-easy effect, verified the success of categorization of questions into three levels of difficulty, and showed that gender was not associated with overconfidence in the developed instrument. The average group overconfidence measures obtained from both experimental phases did not differ from each other significantly. Instrument’s internal consistency was found to be good and acceptable for the use in social research. Compared to the tests used in the foregoing economic experiments, the obtained test is believed to result in the improvement of the overconfidence measurement quality.

• Item Type: MPRA Paper
• Language: English
• Keywords: overconfidence; instrument development; experiment
• Subjects: C - Mathematical and Quantitative Methods > C4 - Econometric and Statistical Methods: Special Topics > C43 - Index Numbers and Aggregation; C - Mathematical and Quantitative Methods > C9 - Design of Experiments > C99 - Other; ; C - Mathematical and Quantitative Methods > C8 - Data Collection and Data Estimation Methodology ; Computer Programs > C80 - General

References

Adams, J. K. (1957). A confidence scale defined in terms of expected percentages. The American Journal of Psychology, Vol. 70(3), p. 432-436.

Adams, P. A., Adams, J. K., (1958), Training in confidence-judgments. The American Journal of Psychology, Vol. 71(4), p. 747-751.

Allwood, C.M., Granhag, P. A., Jonsson, A. C., (2006), Child witnesses’ metamemory realism. Scandinavian Journal of Psychology, Vol. 47(6), p 461-447.

Alpert, M., Raiffa, H., (1982), A progress report on the training of probability assessors. In Kahneman, D., Slovic, P., Tversky, A. (Eds.), Judgment under Uncertainty: Heuristics and Biases, Cambridge University Press, p. 294-305.

Arkes, H. R., Christensen, C., Lai, C., Blumer, C., (1987), Two methods of reducing overconfidence. Organizational Behavior and Human Decision Processes, Vol. 39, p. 133-144.

Barber, B., Odean, T., (2001), Boys will be boys: Gender, overconfidence and common stocks investments. Quarterly Journal of Economics, Vol. 116, p. 261-292.

Barber, B. M., Odean, T., (2002), Online investors: do the slow die first? Review of Financial Studies, Vol. 15(2), p. 455-487.

Bar-Hillel, M., (2001), Subjective probability judgments. In Smelser, N.J. and Baltes, D.B. (Eds.), International Encyclopedia of the Social & Behavioral Sciences, Amsterdam: Elsevier Science Ltd., p. 15247-15251.

Bar-Tal Y., Sarid A., Kishon-Rabin L., (2001), A test of overconfidence phenomenon using audio signals. The Journal of General Psychology, Vol. 128(1), p. 76-80.

Baron, R. A., (2000), Psychological Perspectives on Entrepreneurship: Cognitive and Social Factors in Entrepreneurs’ Success. Current Directions in Psychological Science, Vol. 9(1), p. 15-18.

Benos, A., (1998), Aggressiveness and Survival of Overconfident Traders. Journal of Financial Markets, Vol. 1, p. 353-383.

Bernard V. L., Thomas J., (1989), Post-earnings-announcement drift: delayed price response or risk premium. Journal of Accounting Research, Vol. 27, p. 1–36.

Bernard, V. L., Thomas J., (1990), Evidence that stock prices do not fully reflect the implications of current earnings for future earnings. Journal of Accounting and Economics, Vol. 13(4), p. 305–340.

Biais, B., Hilton, D., Mazurier, K., Pouget, S., (2005), Judgmental overconfidence, self-monitoring and trading performance in an experimental financial market. Review of economic studies, Vol. 72(2), p. 287-312.

Blavatsky, P., (2009), Betting on own knowledge: Experimental test of overconfidence. Journal of Risk and Uncertainty, Vol. 38, p. 39-49.

Bohner, G., Rank, S., Reinhard, M. A., Einwiller, S., Erb, H. P., (1998), Motivational determinants of systematic processing: expectancy moderates effects of desired confidence on processing effort. European Journal of Social Psychology, Vol. 28, p. 185-206.

Caballé, J., Sákovics J., (2003), Speculating against an overconfident market. Journal of Financial Markets, Vol. 6, p. 199-225.

Cambridge, R. M., Shreckengost, R. C., (1978), Are you sure? The subjective probability assessment test. Langley, VA: Office of Training, Central Intelligence Agency.

Camerer, C., (1995), Individual decision making. In Kagel, J. H., Roth, A. (Eds.), Handbook of experimental economics, Princeton University press, p. 587-703.

Clarke, F. R., (1960), Confidence Ratings, Second-Choice Responses, and Confusion Matrices in Intelligibility Tests. Journal of the Acoustical Society of America, Vol. 32(1), p. 35-46.

Conger, R. F., Wolstein, Ch. R., (2004), Managing overconfidence in pricing. Emphasis, Vol. 2, p. 10-13.

Daniel, K., Hirshleifer D., Subrahmanyam A., (1998), Investor psychology and security market under-and overreactions. Journal of Finance, Vol. 53(6), p 1839-1885.

Daniel, K. D, Hirshleifer, D., Subrahmanyam, A., (2001), Overconfidence, arbitrage, and equilibrium asset pricing. Journal of Finance, Vol. 56 (3), p. 921–965.

Deaves, R., Lüders, E., Luo, G. Y., (2009), An Experimental Test of the Impact of Overconfidence and Gender on Trading activity. Review of finance, Vol. 13(3), p. 555–575.

DeCoster, J., (2005), Scale Construction Notes. Retrieved on 04.30.2010 from http://www.stat-help.com/notes.html

De Bondt, W. F. M., Thaler R., (1984), Does the Stock Market Overreact? Journal of Finance, Vol. 40(3), p. 793-805.

De Long, J. B., Shleifer, A., Summers, L. H., Waldmann, R. J., (1991), The Survival of Noise Traders in Financial Markets. The Journal of Business, Vol. 64(1), p. 1-19.

Fenton-O’Creevy, M., Nicholson, N., Soane, E., Willman, P., (2003), Trading on Illusions: Unrealistic Perceptions of Control and Trading Performance. Journal of Occupational and Organizational Psychology, Vol. 76, p. 53–68.

Fischhoff, B., Slovic, P., Lichtenstein, S., (1977), Knowing with certainty: The appropriateness of extreme confidence. Journal of Experimental Psychology: Human Perception and Performance, Vol. 3, p. 552-564.

Fischhoff, B., (1982), Debiasing. In Kahneman, D., Slovic, P., Tversky, A. (Eds.), Judgment under Uncertainty: Heuristics and Biases, Cambridge University Press, p. 422-444.

Gervais, S., Odean, T., (2001), Learning to be Overconfident. Review of Financial Studies, Oxford University Press for Society for Financial Studies, Vol. 14(1), p. 1-27.

Gigerenzer, G., Hoffrage, U., Kleinbölting, H., (1991), Probabilistic mental models: A Brunswikian theory of confidence. Psychological Review, Vol. 98, p. 506-528.

Glaser, M., Langer, T., Weber, M., (2005), Overconfidence of Professionals and Lay Men: Individual Differences within and between Tasks? Working Paper, Mannheim.

Glaser, M., Weber, M., (2007), Overconfidence and trading volume. The Geneva Risk and Insurance Review, Vol. 32(1), p. 1-36.

Hazard, T. H., Peterson, C. R., (1973), Odds versus probabilities for categorical events. Tech. Rep. No. 73-2, McLean, VA: Decisions and Designs, Inc.

Hirshleifer D., Subrahmanyam, A., Titman S., (1994), Security analysis and trading patterns when some investors receive information before others. Journal of Finance, Vol. 49(5), p. 1665-1698.

Hoelzl, E., Rustichini, A., (2005), Overconfident: do you put your money on it? Economic Journal, Vol. 115, p. 305-318.

Hynes, M., Vanmarcke, E., (1976), Reliability of embankment performance predictions. Proceedings of the ASCE Engineering Mechanics Division Specialty Conference, Waterloo, Ontario, Canada: University of Waterloo Press.

Johnson, D. D. P., McDermott, R., Barrett E. S., Cowden, J., Wrangham R., McIntyre, M.H., Rosen, S. P., (2006), Overconfidence in wargames: experimental evidence on expectations, aggression, gender and testosterone. Proceedings: Biological Sciences, Vol. 273(1600), p. 2513-2520.

Kahneman D., Riepe, M. W., (1998), Aspects of Investor Psychology. Journal of Portfolio Management, Vol. 24(4), p. 52-65.

Keasey, K., Watson, R., (1989), Consensus and accuracy in accounting studies of decision making: A note on a new measure of consensus. Accounting, Organizations and Society, Vol. 14, p. 337-345.

Kim, A. K., Nofsinger, J. R., (2003), The behavior and performance of individual investors in Japan. Working paper.

Kirchler, E., Maciejovsky, B., (2002), Simultaneous over- and underconfidence: evidence from experimental asset markets. Journal of Risk and Uncertainty, Springer, Vol. 25(1), p. 65-85.

Klayman, J., Soll., J. B., Gonzáles-Vallejo, C., Barlas, S., (1999), Overconfidence: it depends on how, what, and whom you ask. Organizational Behavior and Human Decision Processes, Vol. 79(3), p. 216-247.

Koriat A, Lichtenstein S, Fischhoff B., (1980), Reasons for confidence. Journal of Experimental Psychology: Human Learning and Memory, Vol. 6, p. 107-18.

Kyle, A., Wang, F. A., (1997), Speculation duopoly with agreement to disagree: can overconfidence survive the market test? Journal of Finance, Vol. 52(5), p. 2073-2090.

Lakonishok, J., Shleifer, A., Vishny, R. W., (1992), The impact of institutional trading on stock prices. Journal of Financial Economics, Vol. 32(1), p. 23-43.

Langer, E. J., (1982), The illusion of control. In: Kahneman, D., Slovic, P., Tversky A. (Eds.), Judgment under uncertainty: heuristics and biases, Cambridge University Press, p. 231-338.

Loughran, T., Ritter, J. R., (1995), The new issues puzzle. Journal of Finance, Vol. 50(1), p. 23-51.

Lichtenstein, S., Fischhoff, B., (1980), Training for calibration. Organizational Behavior and Human Performance, Vol. 26, p. 149-171.

Lichtenstein, S., Fischhoff, B., (1981), The effects of gender and instructions on calibration (Decision Research Report 81-5). Eugene OR: Decision Research.

Lichtenstein, S., Fischhoff, B., and Phillips, L. D., (1977), Calibration of probabilities: The state of the art. In Jungermann, H., deZeeuw, G. (Eds.), Decision making and change in human affairs, Amsterdam: D. Reidel, p. 275-324.

Lichtenstein, S., Fischhoff, B., Phillips, L. D., (1982), Calibration of probabilities: the state of the art to 1980. Judgment under Uncertainty: Heuristics and Biases. In Kahneman, D., Slovic, P., Tversky, A. (Eds.), Judgment under Uncertainty: Heuristics and Biases, Cambridge University Press, p. 306-334.

Menkhoff, L., Schmidt, U., Brozynski, T., (2006), The impact of experience on risk taking, overconfidence, and herding of fund managers: Complementary survey evidence. European Economic Review, Vol. 50(7), p. 1753-1766

Menkhoff, L., Schmeling, M., Schmidt, U., (2006), Does Professionalism Consistently Affect Portfolio Biases? Working Paper.

Michaely, R., Thaler, R. H., Womack, K. L., (1995), Price Reactions to Dividend Initiations and Omissions: Overreaction or Drift? Journal of Finance, Vol. 50(2), p. 573-608.

Moore, P. G., (1977), The manager's struggle with uncertainty. Journal of the Royal Statistical Society, Vol. 140, p. 129-165.

Moss, S., Patel, P., Prosser, H., Goldber, D., Simpson, N., Rowe, S., Lucchino, R., (1993), Psychiatric morbidity in older people with moderate and severe learning disability. I: Development and reliability of the patient interview (PAS–ADD). British Journal of Psychiatry, Vol. 163, p. 471-480.

Neale, M. A., Bazerman, M. H., (1990), Cognition and rationality in negotiation. New York: The Free Press, p. 240.

Nickerson, R. S., McGoldrick, C. C., (1965), Confidence ratings and level of performance on a judgmental task. Perceptual & Motor Skills, Vol. 20, p. 311-316.

Nöth, M., Weber, M., (2003), Information Aggregation with Random Ordering: Cascades and Overconfidence. The Economic Journal, Vol. 113, p. 166-189.

Oberlechner, T., Osler, C.L., (2003), Overconfidence in currency markets. Working Paper.

Odean, T., (1998), Volume, volatility, price and profit when all traders are above average. Journal of Finance, Vol. 53(6), p. 1887-1934.

Odean, T., (1999), Do investors trade too much? American Economic Review, Vol. 89(5), p. 1278-1298.

Oechssler, J., Schmidt, C., Schnedler, W., (2007), Asset bubbles without dividends - an experiment. Working Paper Nr. 07-01, University of Mannheim.

Oskamp, S., (1962), The relationship of clinical experience and training methods to several criteria of clinical prediction. Psychological Monographs, Vol. 76(547), p. 1-28.

Pitz, G.F., (1974), Subjective probability distributions for imperfectly known quantities. In Gregg, L. W. (Ed.), Knowledge and Cognition. New York: Wiley, p. 29-41.

Powel, W. D., Bolich, C., (1993), Changing predictions based on knowledge of past performance in preschool and young grades-school. Child Study Journal, Vol. 23(3), p. 209.

Pulford, D. B., (1996), Overconfidence in Human Judgment, PhD thesis. Department of Psychology, University of Leicester.

Pulford, B. D., Colman, A. M., (1997), Overconfidence: Feedback and Item Difficulty Effects. Personality and Individual Differences, Vol. 23(1), p. 125-133.

Ronis, D. L., Yates, J. F., (1987), Components of probability judgment accuracy: Individual consistency and effects of subject matter and assessment method. Organizational Behavior and Human Decision Processes, Vol. 40, p. 193-218.

Russo, J. E., Schoemaker, P. J., (1992), Managing overconfidence. Sloan Management Review, Vol. 33, p. 7-17.

Scheinkman, J. A., Xiong, W., (2003), Overconfidence and speculative bubbles. Journal of Political Economy, Vol. 111, p. 1183-1219.

Shiller, R. J., (2000), Irrational Exuberance. Princeton, N.J.: Princeton Univ. Press, 344 p.

Sieber, J. E., (1979), Confidence estimates on the correctness of constructed and multiple-choice responses. Contemporary Educational Psychology, Vol. 4, p. 272-287.

Sniezek, J. A., Paese, P. W., Switzer, F. S., (1990), The effect of choosing on confidence in choice. Organizational Behavior and Human Decision Processes, Vol. 46, p. 264-282.

Staël von Holstein, C. A. S., (1972), Probabilistic forecasting: An experiment related to the stock market. Organizational Behavior and Human Performance, Vol. 8, p. 139-158.

Statman, M., Thorley, S., Vorkink, K., (2006), Investor overconfidence and trading volume. Review of Financial Studies, Vol. 19, p. 1531–1565.

Stotz, O., von Nitzsch, R., (2005), The perception of control and the levels of overconfidence: evidence from analyst earnings estimates and prices targets. The Journal of Behavioral Finance, Vol. 6(3), p. 121-128.

Sümer, N., Özkan, T., Lajunen, T., (2006), Asymmetric relationship between driving and safety skills. Accident Analysis and Prevention, Vol. 38(4), p. 703-711.

Svenson, O., (1981), Are we all less risky and more skilful than our fellow drivers? Acta Psychologica, Vol. 47(2), p. 143-148.

Taylor, S. E., and Brown, J. D., (1988), Illusion and well-being: A social psychological perspective on mental health. Psychological Bulletin, Vol. 103, p. 193-210.

Tversky, A., Kahneman, D, (1982), Judgment under Uncertainty: Heuristics and Biases. In Kahneman, D., Slovic, P., Tversky, A. (Eds.), Judgment under Uncertainty: Heuristics and Biases, Cambridge University Press, p. 3-20.

Wagenaar, W. A., Keren, G., (1986), Does the expert know? The reliability of predictions and confidence ratings of experts. In E. Hollnagel, G. Maneini, and D. D. Woods (Eds.), Intelligent decision support in process environments, Berlin: Springer, p. 87-107.

Weinstein, N. D., (1980), Unrealistic optimism about future life events. Journal of Personality and Social Psychology, Vol. 39, p. 806-820.

Zakay, D., Glicksohn, J., (1992), Overconfidence in a multiple-choice test and its relationship to achievement. Psychological Record, Vol. 42(4), p 519-524.