Internet CV surveys – a cheap, fast way to get large samples of biased values?

Lindhjem, Henrik and Navrud, Ståle (2008): Internet CV surveys – a cheap, fast way to get large samples of biased values?

Abstract

With the current growth in broadband penetration, Internet is likely to be the data collection mode of choice for contingent valuation (CV) and stated preference research in the not so distant future. However, little is known about how this survey mode may influence data quality and welfare estimates. In a controlled field experiment as part of a large national CV survey estimating willingness to pay (WTP) for biodiversity protection plans, we assign two groups sampled from the same panel of respondents, either to an Internet or in-person interview mode. Our design is better able than previous mode comparison studies to isolate measurement effects from sample composition effects. Looking in particular for indications of social desirability bias and satisficing (shortcutting the response process) we find little evidence in our data. We find that the extent of “don’t know”, zeros and protest responses to the WTP question (with a payment card) is very similar between modes. Mean WTP is somewhat higher in the interview sample, though we cannot reject equality on the 10 percent level. We also consider equivalence, i.e. whether the WTP difference is larger than a practically trivial predetermined bound. We can reject that the difference is larger than 30 percent, but fail to reject an equivalency bound of 20 percent on the 10 percent level. Results are quite encouraging for the use of Internet as values do not seem to be significantly different or substantially biased compared to in-person interviews.

• Item Type: MPRA Paper
• Language: English
• Keywords: Internet; contingent valuation; interviews; mode; willingness to pay
• Subjects: H - Public Economics > H4 - Publicly Provided Goods > H41 - Public Goods; Q - Agricultural and Natural Resource Economics ; Environmental and Ecological Economics > Q5 - Environmental Economics > Q51 - Valuation of Environmental Effects

References

Alvarez, R. M., R. P. Sherman and C. VanBeselaere (2003), 'Subject acquisition for Web-based surveys', Polit. Anal. 11(1): 23-43.

Andreoni, J. (1990), 'Impure Altruism And Donations To Public-Goods - A Theory Of Warm-Glow Giving', Econ. J. 100(401): 464-477.

Arlinghaus, R. and T. Mehner (2003), 'Management and Ecological Note: Testing the reliability and construct validity of a simple and inexpensive procedure to measure the use value of recreational fishing', Fisheries Management and Ecology 11: 61-64.

Arrow, K. J., R. Solow, E. Leamer, P. Portney, R. Radner and H. Schuman (1993), 'Report of the NOAA Panel on Contingent Valuation', Federal Register 58: 4601-4614.

Banzhaf, H. S., D. Burtraw, D. Evans and A. Krupnick (2006), 'Valuation of natural resource improvements in the Adirondacks', Land Economics 82(3): 445-464.

Bateman, I., M. Cole, P. Cooper, S. Georgiou, D. Hadley and G. L. Poe (2004), 'On visible choice sets and scope sensitivity', Journal of Environmental Economics and Management 47: 71-93.

Bateman, I. J., D. Burgess, G. H. Hutchinson and D. I. Matthews (2008), 'Learning design contingent valuation (LDCV): NOAA guidelines, preference learning and coherent arbitrariness', Journal of Environmental Economics and Management 55: 127-141.

Bateman, I. J., R. T. Carson, B. Day, W. M. Hanemann, N. Hanley, T. Hett, M. Jones-Lee, G. Loomes, S. Mourato, E. Ozdemiroglu, D. W. Pearce, R. Sugden and T. Swanson (2002), Economic Valuation with Stated Preference Techniques: A Manual. Edward Elgar Publishing, Cheltenham, 480pp.

Bateman, I. J. and J. Mawby (2004), 'First impressions count: interviewer appearance and information effects in stated preference studies', Ecological Economics 49(1): 47-55.

Berrens, R. P., A. K. Bohara, H. Jenkins-Smith, C. Silva and D. L. Weimer (2003), 'The advent of Internet surveys for political research: A comparison of telephone and Internet samples', Polit. Anal. 11(1): 1-22.

Berrens, R. P., A. K. Bohara, H. C. Jenkins-Smith, C. L. Silva and D. L. Weimer (2004), 'Information and effort in contingent valuation surveys: application to global climate change using national internet samples', Journal of Environmental Economics and Management 47(2): 331-363.

Blamey, R. K., J. W. Bennett and M. D. Morrison (1999), 'Yea-Saying in Contingent Valuation Surveys', Land Economics 75(1): 126-141.

Blomquist, G. C. and J. C. Whitehead (1998), 'Resource quality information and validity of willingness to pay in contingent valuation', Resour. Energy Econ. 20(2): 179-196.

Boyle, K. J. (2003), 'Contingent valuation in practice', in P. A. Champ, K. J. Boyle and T. C. Brown, eds, A primer on nonmarket valuation. Kluwer Academic Publishers.

Boyle, K. J. and J. C. Bergstrom (1999), 'Doubt, doubt, and doubters: The genesis of a new research agenda?' in I. Bateman and K. G. Willis, eds, Valuing Environmental Preferences. Oxford University Press.

Cameron, T. A. and D. D. Huppert (1989), 'Ols Versus Ml Estimation Of Non-Market Resource Values With Payment Card Interval Data', Journal of Environmental Economics and Management 17(3): 230-246.

Couper, M. P. (2000), 'Web surveys - A review of issues and approaches', Public Opin. Q. 64(4): 464-494.

Couper, M. P. (2005), 'Technology trends in survey data collection', Soc. Sci. Comput. Rev. 23(4): 486-501.

Damschroder, L. J., P. A. Ubel, J. Riis and D. M. Smith (2007), 'An alternative approach for eliciting willingness-to-pay: A randomized Internet trial', Judgment and Decision Making 2(2): 96-106.

Davis, J. (2004), 'Assessing Community Preferences for Development Projects: Are Willingness-to-Pay Studies Robust to Mode Effects?' World Development 32(4): 655-672.

DeMaio, T. J. (1984), 'Social desirability and survey measurement: A review', in C. F. Turner and E. Martin, eds, Surveying subjective phenomena: Volume 2. New York: Russel Sage.

Denscombe, M. (2006), 'Web-based questionnaires and the mode effect - An evaluation based on completion rates and data contents of near-identical questionnaires delivered in different modes', Soc. Sci. Comput. Rev. 24(2): 246-254.

DeShazo, J. R. and G. Fermo (2002), 'Designing choice sets for stated preference methods: The effects of complexity on choice consistency', J.Environ.Econ.Manage. 44(1): 123-143.

Dickie, M., S. Gerking and W. L. Goffe (2007). Valuation of Non-Market Goods Using Computer-Assisted Surveys: A Comparison of Data Quality from Internet and RDD Sample. Presented at European Association of Environmental and Resource Economists, Thessaloniki, Greece, June 2007

Dillman, D. (2000), Mail and internet surveys: the tailored design method. John Wiley & Sons, Inc.

Dillman, D. and J. M. Bowker (2001), 'The WEB questionnaire challenge to survey methodologists', in U.-D. Reips and M. Bosnjak, eds, Dimensions of Internet Science. Lengerich, Germany: Pabst Science Publishers, pp. 159-178.

Dillman, D. A. and J. D. Smyth (2007), 'Design Effects in the Transition to Web-based Surveys ', American Journal of Preventive Medicine 32: S90-S96.

Edwards, S. F. and G. D. Anderson (1987), 'Overlooked biases in contingent valuation surveys: Some considerations', Land Economics 63(2): 168-178.

Efron, B. (1997), 'Bootstrap methods: Another look at the jacknife', Annals of Statistics 7: 1-26.

Epstein, J., W. D. Klinkenberg, D. Wiley and L. McKinley (2001), 'Insuring sample equivalence across internet and paper-and-pencil assessments', Computers in Human Behavior 17: 339-346.

Ethier, R. G., G. L. Poe, W. D. Schulze and J. Clark (2000), 'A comparison of hypothetical phone and mail contingent valuation responses for green-pricing electricity programs', Land Econ. 76(1): 54-67.

Fricker, S., M. Galesic, R. Tourangeau and T. Yan (2005), 'An experimental comparison of web and telephone surveys', Public Opinion Quarterly 69(3): 370-392.

Gabaix, X., D. Laibson and G. Moloche (2003). The allocation of attention: theory and evidence, Working Paper, Harvard University

Gilovich, T., D. Griffin and D. Kahneman (2002), Heuristics and biases : the psychology of intuitive judgement Cambridge University Press, Cambridge.

Green, C. and S. Tunstall (1999), 'A psychological perspective', in I. Bateman and K. G. Willis, eds, Valuing environmental preferences: Theory and practice of the contingent valuation method in the US, EU, and developing countries. Oxford University Press.

Groves, R. M., F. J. Fowler jr, M. P. Couper, J. M. Lepkowski, E. Singer and R. Tourangeau (2004), Survey Methodology. Wiley.

Hanley, N. (1989), 'Valuing rural recreation benefits: An empirical comparison of two approaches', Journal of Agricultural Economics 40(3): 361-74.

Harpman, D. A., M. P. Welsh and E. W. Sparling (2004), 'Unit non-response bias in the interval data model', Land Economics 80(3): 448-462.

Heckman, J. J. (1979), 'Sample selection bias as a specification model', Econometrica 47(1): 153-161.

Hoehn, J. P., F. Lupi and M. D. Kaplowitz (2004). Internet-based Stated Choice Experiments in Ecosystem Mitigation: Methods to Control Decision Heuristics and Biases

Holbrook, A. L., M. C. Green and J. Krosnick (2003), 'Telephone versus face-to-face interviewing of national probability samples with long questionnaires: Comparison of respondent satisficing and social desirability response bias', Public Opinion Quarterly 67: 79-125.

Holtgraves, T. (2004), 'Social Desirability and Self-Reports: Testing Models of Socially Desirable Responding ', Personality and Social Psychology Bulletin 30(2): 161-172.

Hudson, D., L. Seah, D. Hite and T. C. Haab (2004), 'Telephone presurveys, self-selection, and non-response bias to mail and internet surveys in economic research', Applied Economics Letters 11(237-240).

Haab, T. C. and K. E. McConnell (2002), Valuing Environmental and Natural Resources: The econometrics of non-market valuation. Edward Elgar.

Johnston, R. J. (2007), 'Choice experiments, site similarity and benefits transfer', Environmental & Resource economics 38: 331-351.

Johnston, R. J., E. Y. Besedin, R. Iovanna, C. J. Miller, R. F. Wardwell and M. H. Ranson (2005), 'Systematic variation in willingness to pay for aquatic resource improvements and implications for benefit transfer: a meta-analysis', Canadian Journal of Agricultural Economics 53(2-3): 221-248.

Jäckle, A., C. Roberts and P. Lynn (2006). Telephone versus Face-to-Face Interviewing: Mode Effects on Data Quality and Likely Causes. Report on Phase II of the ESS-Gallup Mixed Mode Methodology Project,

Kahneman, D. and A. Tversky, Eds. (2000), Choices, Values and Frames, Cambridge University Press.

Karp, J. A. and D. Brockington (2005), 'Social desirability and response validity: A comparative analysis of overreporting voter turnout in five countries ', Journal of Politics 67(3): 825-840.

Kramer, R. A. and J. I. Eisen-Hecht (2002), 'Estimating the economic value of water quality protection in the Catawba River basin', Water Resources Research 38(9).

Kristofersson, D. and S. Navrud (2005), 'Validity Tests of Benefit Transfer – Are We Performing the Wrong Tests?' Environmental and Resource Economics 30: 279-286.

Kristofersson, D. and S. Navrud (2007), 'Can Use and Non-Use Values be Transferred Across Countries?' in S. Navrud and R. Ready, eds, Environmental Value Transfer: Issues and Methods. Dordrecht, The Netherlands: Springer.

Krosnick, J. (1991), 'Response strategies for coping with the cognitive demands of attitude measures in surveys', Applied Cognitive Psychology 5: 213-36.

Krosnick, J. and L. Chang (2001). A comparison of the random digit dialing telephone survey methodology with internet survey methodology as implemented by knowledge networks and Harris interactive, Ohio State University

Legget, C. G., N. S. Kleckner, K. Boyle, J. W. Duffield and R. C. Micthel (2003), 'Social Desirability Bias in Contingent Valuation Surveys Administered Through In-Person Interviews', Land Economics 79(4): 561–575.

Li, H., R. P. Berrens, A. K. Bohara, H. C. Jenkins-Smith, C. L. Silva and D. L. Weimer (2005), 'Testing for budget constraint effects in a national advisory referendum survey on the Kyoto protocol', J. Agric. Resour. Econ. 30(2): 350-366.

Lienhoop, N. and D. MacMillan (2007), 'Contingent Valuation: Comparing Participant Performance in Group-Based Approaches and Personal Interviews', Environmental Values 16: 209-232.

Lindberg, K., R. L. Johnson and R. P. Berrens (1997), 'Contingent valuation of rural tourism development with tests of scope and mode stability', J. Agric. Resour. Econ. 22(1): 44-60.

Lindhjem, H. (2007), '20 Years of stated preference valuation of non-timber benefits from Fennoscandian forests: A meta-analysis', Journal of Forest Economics 12(4): 251-277.

Lindhjem, H. and S. Navrud (2008). Asking for Individual or Household Willingness to Pay for Environmental Goods: Implication for aggregate welfare measures Department of Economics and Resource Management, Norwegian University of Life Sciences

List, J. A., A. K. Bohara and J. Kerkvliet (2004), 'Examining the Role of Social Isolation on Stated Preferences', American Economic Review 94(3): 741-752.

Loomis, J. and M. King (1994), 'Comparison Of Mail And Telephone-Mail Contingent Valuation Surveys', J. Environ. Manage. 41(4): 309-324.

Loomis, J., J. Miller, A. Gonzales-Caban and P. A. Champ (2006), 'Testing the Convergent Validity of Videotape Survey Administration and Phone Interviews in Contingent Valuation', Society and Natural Resources 19(4): 367-375.

Loureiro, M. L. and J. Lotade (2005), 'Interviewer Effects on the Valuation of Goods with Ethical and Environmental Attributes', Environmental & Resource economics 30: 49-72.

MacMillan, D., N. Hanley and N. Lienhoop (2006), 'Contingent valuation: Environmental polling or preference engine?' Ecological Economics 60(1): 299-307.

Maguire, K. B., G. Shiferaw and L. O. Taylor (2002). Mode and subject pool effects in contingent valuation surveys using hurdle models. Working paper

Malhotra, N. and J. Krosnick (2007), 'The Effect of Survey Mode and Sampling on Inferences about Political Attitudes and Behavior: Comparing the 2000 and 2004 ANES to Internet Surveys with Nonprobability Samples', Political Analysis: 286-323.

Mannesto, G. and J. B. Loomis (1991), 'Evaluation Of Mail And In-Person Contingent Value Surveys - Results Of A Study Of Recreational Boaters', J. Environ. Manage. 32(2): 177-190.

Marta-Pedroso, C., H. Freitas and T. Domingos (2007), 'Testing for the survey mode effect on contingent valuation data quality: a case study of web based versus in-person interviews.' Ecological Economics 62: 388-398.

McFadden, D. (1999), 'Rationality for economists?' J. Risk Uncertain. 19(1-3): 73-105.

Messonier, M. L., J. C. Bergstrom, C. M. Cornwell, R. J. Teasley and H. K. Cordell (2000), 'Survey Response-Related Biases in Contingent Valuation: Concepts, Remedies, and Empirical Application to Valuing Aquatic Plant Management', American Journal of Agricultural Economics 83: 438-450.

Mitchell, R. C. and R. T. Carson (1989), Using Surveys to Value Public Goods: The Contingent Valuation Method. Washington DC: Resources for the Future.

Navrud, S. and G. Pruckner (1997), 'Environmental Valuation - To Use or Not to Use? A Comparative Study of the United States and Europe', Environmental and Resource Economics 10(1): 1-26.

OECD (2008). OECD Broadband Portal. http://www.oecd.org/sti/ict/broadband

Olsen, S. B. (2007). Internet versus mail: Are stated preferences affected by the mode of sampling in a choice experiment? Presented at European Association of Environmental and Resource Economists, Thessaloniki, Greece, June 2007.

Payne, J. W., J. R. Bettman and D. A. Schade (1999), 'Measuring Constructed Preferences: Towards a Building Code', Journal of Risk and Uncertainty 19(1-3): 243-270.

Roger, J. L., K. I. Howard and J. T. Vessey (1993), 'Using significance tests to evaluate equivalence between two experimental groups', Psychological Bulletin 113(3): 553-565.

Rosenberger, R. S. and J. B. Loomis (2000), 'Using meta-analysis for benefit transfer: In-sample convergent validity tests of an outdoor recreation database', Water Resources Research 36(4): 1097-1107.

Sanders, D., H. D. Clarke, M. C. Stewart and P. Whiteley (2007), 'Does Mode Matter For Modeling Political Choice? Evidence From the 2005 British Election Study', Political Analysis: 257-285.

Schkade, D. and D. Kahneman (1998), 'Does living in California make people happy? A focusing illusion in judgments of life satisfaction', Psychol. Sci. 9: 340-346.

Schulze, W., G. McClelland, D. Waldman and J. H. Lazo (1996), 'Sources of bias in contingent valuation', in D. J. Bjornstad and J. Kahn, eds, The contingent valuation of environmental resources: Methodological resources and research needs. Edward Elgar.

Schuman, H. (1996), 'The sensitivity of CV outcomes to CV survey methods', in D. J. Bjornstad and J. Kahn, eds, The contingent valuation of environmental resources: Methodological issues and research needs. Edward Elgar.

Schwappach, D. L. B. and T. J. Strasman (2006), '“Quick and dirty numbers”? The reliability of a stated-preference technique for the measurement of preferences for resource allocation', Journal of Health Economics 25: 432-448.

Shogren, J. F. (2005), 'Experimental methods and valuation', in K.-G. Maler and J. R. Vincent, eds, Handbook of Environmental Economics. . Amsterdam: North Holland.

Smith, V. K. (2000), 'JEEM and non-market valuation: 1974-1998', J.Environ.Econ.Manage. 39(3): 351-374.

Stanton, J. (1998), 'An empirical assessment of data collection using the internet', Personnel Psychology 51: 709-725.

Statistics Norway (2008). The Internet Poll, 4th Quarter 2007

Thurston, H. W. (2006), 'Non-market valuation on the internet', in A. Alberini and J. Kahn, eds, Handbook on contingent valuation. Edward Elgar.

Tourangeau, R., M. P. Couper and F. Conrad (2004), 'Spacing, position, and order - Interpretive heuristics for visual features of survey questions', Public Opin. Q. 68(3): 368-393.

Tourangeau, R., M. P. Couper and F. Conrad (2007), 'Color, labels, and interpretive heuristics for response scales', Public Opinion Quarterly 71(1): 91-112.

Tourangeau, R., L. J. Rips and K. A. Rasinski (2000), The psychology of survey response. Cambridge: Cambridge University Press.

Tsuge, T. and T. Washida (2003), 'Economic valuation of the Seto Inland Sea by using an Internet CV survey', Marine Pollution Bulletin 47: 230-236.

Urama, K. C. and I. Hodge (2006), 'Participatory environmental education and willingness to pay for river basin management: Empirical evidence from Nigeria', Land Economics 82(4): 542-561.

Welling, P. G., F. L. S. Tse and S. V. Dighe (1991), Pharmaceutical Bioequivalence. New York: Marcel Dekker.

Whittaker, D., J. J. Vaske, M. P. Donnelly and D. S. DeRuiter (1998), 'Mail versus telephone surveys: Potential biases in expenditure and willingness-to-pay', Journal of Park and Recreation Administration 16(3): 15-30.

Whittington, D., V. K. Smith, A. Okorafor, A. Okore, J. L. Liu and A. McPhail (1992), 'Giving Respondents Time To Think In Contingent Valuation Studies - A Developing-Country Application', J.Environ.Econ.Manage. 22(3): 205-225.