Can cheap panel-based internet surveys substitute costly in-person interviews in CV surveys?

Lindhjem, Henrik and Navrud, Ståle (2010): Can cheap panel-based internet surveys substitute costly in-person interviews in CV surveys?

Abstract

With the current growth in broadband penetration, Internet is likely to be the data collection mode of choice for stated preference research in the not so distant future. However, little is known about how this survey mode may influence data quality and welfare estimates. In a first controlled field experiment to date as part of a national contingent valuation (CV) survey estimating willingness to pay (WTP) for biodiversity protection plans, we assign two groups sampled from the same panel of respondents either to an Internet or in-person (in-house) interview mode. Our design is better able than previous studies to isolate measurement effects from sample composition effects. We find little evidence of social desirability bias in the in-person interview setting or satisficing (shortcutting the response process) in the Internet survey. The share of “don’t knows”, zeros and protest responses to the WTP question with a payment card is very similar between modes. Equality of mean WTP between samples cannot be rejected. Considering equivalence, we can reject that mean WTP from the in-person sample is more than 30% higher. Results are quite encouraging for the use of Internet in CV as stated preferences do not seem to be significantly different or biased compared to in-person interviews.

• Item Type: MPRA Paper
• Language: English
• Keywords: Internet; contingent valuation; interviews; survey mode; willingness to pay
• Subjects: H - Public Economics > H4 - Publicly Provided Goods > H41 - Public Goods; Q - Agricultural and Natural Resource Economics ; Environmental and Ecological Economics > Q5 - Environmental Economics > Q51 - Valuation of Environmental Effects

References

Alvarez R M, Sherman R P and VanBeselaere C (2003) Subject acquisition for Web-based surveys. Polit. Anal. 11(1): 23-43

Amiran E Y and Hagen D A (2010) The Scope Trials: Variation in Sensitivity to Scope and WTP with Directionally Bounded Utility Functions. Journal of Environmental Economics and Management 59(3): 293-301

Arrow K J, Solow R, Leamer E, Portney P, Radner R and Schuman H (1993) Report of the NOAA Panel on Contingent Valuation. Federal Register 58: 4601-4614

Banzhaf H S, Burtraw D, Evans D and Krupnick A (2006) Valuation of natural resource improvements in the Adirondacks. Land Economics 82(3): 445-464

Bateman I, Cole M, Cooper P, Georgiou S, Hadley D and Poe G L (2004) On visible choice sets and scope sensitivity. Journal of Environmental Economics and Management 47: 71-93

Bateman I J, Burgess D, Hutchinson G H and Matthews D I (2008) Learning design contingent valuation (LDCV): NOAA guidelines, preference learning and coherent arbitrariness. Journal of Environmental Economics and Management 55: 127-141

Bateman I J and Mawby J (2004) First impressions count: interviewer appearance and information effects in stated preference studies. Ecological Economics 49(1): 47-55

Berrens R P, Bohara A K, Jenkins-Smith H, Silva C and Weimer D L (2003) The advent of Internet surveys for political research: A comparison of telephone and Internet samples. Polit. Anal. 11(1): 1-22

Berrens R P, Bohara A K, Jenkins-Smith H C, Silva C L and Weimer D L (2004) Information and effort in contingent valuation surveys: application to global climate change using national internet samples. Journal of Environmental Economics and Management 47(2): 331-363

Boyle K J (2003) Contingent valuation in practice. In: P. A. Champ, K. J. Boyle and T. C. Brown (ed), A primer on nonmarket valuation. Kluwer Academic Publishers,

Boyle K J and Bergstrom J C (1999) Doubt, doubt, and doubters: The genesis of a new research agenda? In: I. Bateman and K. G. Willis (ed), Valuing Environmental Preferences. Oxford University Press,

Cameron T A and Huppert D D (1989) Ols Versus Ml Estimation Of Non-Market Resource Values With Payment Card Interval Data. Journal of Environmental Economics and Management 17(3): 230-246

Canavari M, Nocella G and Scarpa R (2005) Stated Willingness-to-Pay for Organic Fruit and Pesticide Ban: An Evaluation Using Both Web-Based and Face-to-Face Interviewing. Journal of Food Products Marketing 11(3): 107-134

Carlsson F (2010) Design of Stated Preference Surveys: Is There More to Learn from Behavioral Economics? Environmental and Resource Economics 46(2): 167-177

Chang L and Krosnick J A (2009) National surveys via RDD telephone interviewing versus the internet comparing sample representativeness and response quality. Public Opinion Quarterly 73(4): 641-678

Couper M P (2000) Web surveys - A review of issues and approaches. Public Opin. Q. 64(4): 464-494

Couper M P (2005) Technology trends in survey data collection. Soc. Sci. Comput. Rev. 23(4): 486-501

Covey J, Robinson A, Jones-Lee M and Loomes G (2010) Responsibility, scale and the valuation of rail safety. Journal of Risk and Uncertainty 40: 85-108

Davis J (2004) Assessing Community Preferences for Development Projects: Are Willingness-to-Pay Studies Robust to Mode Effects? World Development 32(4): 655-672

DeMaio T J (1984) Social desirability and survey measurement: A review. In: C. F. Turner and E. Martin (ed), Surveying subjective phenomena: Volume 2. New York: Russel Sage,

Dickie M, Gerking S and Goffe W L (2007). Valuation of Non-Market Goods Using Computer-Assisted Surveys: A Comparison of Data Quality from Internet and RDD Samples

Dillman D (2000) Mail and internet surveys: the tailored design method. John Wiley & Sons, Inc,

Dillman D and Bowker J M (2001) The WEB questionnaire challenge to survey methodologists. In: U.-D. Reips and M. Bosnjak (ed), Dimensions of Internet Science. Lengerich, Germany: Pabst Science Publishers, pp. 159-178

Dillman D A and Smyth J D (2007) Design Effects in the Transition to Web-based Surveys American Journal of Preventive Medicine 32: S90-S96

Ehmke M D, Lusk J L and List J A (2008) Is Hypothetical Bias a Universal Phenomenon? A Multinational Investigation. Land Economics 84(3): 489-500

Epstein J, Klinkenberg W D, Wiley D and McKinley L (2001) Insuring sample equivalence across internet and paper-and-pencil assessments. Computers in Human Behavior 17: 339-346

Ethier R G, Poe G L, Schulze W D and Clark J (2000) A comparison of hypothetical phone and mail contingent valuation responses for green-pricing electricity programs. Land Econ. 76(1): 54-67

Green C and Tunstall S (1999) A psychological perspective. In: I. Bateman and K. G. Willis (ed), Valuing environmental preferences: Theory and practice of the contingent valuation method in the US, EU, and developing countries. Oxford University Press,

Groves R M, Fowler jr F J, Couper M P, Lepkowski J M, Singer E and Tourangeau R (2004) Survey Methodology. Wiley,

Harrison G W and Lau M I (2009) Risk attitudes, randomization to treatment, and self-selection into experiments. Journal of Economic Behavior & Organization 70: 498-507

Heckman J J (1979) Sample selection bias as a specification model. Econometrica 47(1): 153-161

Holbrook A L, Green M C and Krosnick J (2003) Telephone versus face-to-face interviewing of national probability samples with long questionnaires: Comparison of respondent satisficing and social desirability response bias. Public Opinion Quarterly 67: 79-125

Hudson D, Seah L, Hite D and Haab T C (2004) Telephone presurveys, self-selection, and non-response bias to mail and internet surveys in economic research. Applied Economics Letters 11(237-240)

Jäckle A, Roberts C and Lynn P (2006). Telephone versus Face-to-Face Interviewing: Mode Effects on Data Quality and Likely Causes. Report on Phase II of the ESS-Gallup Mixed Mode Methodology Project,

Karp J A and Brockington D (2005) Social desirability and response validity: A comparative analysis of overreporting voter turnout in five countries Journal of Politics 67(3): 825-840

Kristofersson D and Navrud S (2005) Validity Tests of Benefit Transfer – Are We Performing the Wrong Tests? Environmental and Resource Economics 30: 279-286

Kristofersson D and Navrud S (2007) Can Use and Non-Use Values be Transferred Across Countries? In: S. Navrud and R. Ready (ed), Environmental Value Transfer: Issues and Methods. Dordrecht, The Netherlands: Springer,

Krosnick J (1991) Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology 5: 213-36

Legget C G, Kleckner N S, Boyle K, Duffield J W and Micthel R C (2003) Social Desirability Bias in Contingent Valuation Surveys Administered Through In-Person Interviews. Land Economics 79(4): 561–575

Li H, Berrens R P, Bohara A K, Jenkins-Smith H C, Silva C L and Weimer D L (2005) Testing for budget constraint effects in a national advisory referendum survey on the Kyoto protocol. J. Agric. Resour. Econ. 30(2): 350-366

Li H, Berrens R P, Bohara A K, Jenkins-Smith H C, Silva C L and Weimer L (2004) Telephone versus Internet samples for a national advisory referendum: are the underlying stated preferences the same? Appl. Econ. Lett. 11(3): 173-176

Lienhoop N and MacMillan D (2007) Contingent Valuation: Comparing Participant Performance in Group-Based Approaches and Personal Interviews. Environmental Values 16: 209-232

Lindhjem H (2007) 20 Years of stated preference valuation of non-timber benefits from Fennoscandian forests: A meta-analysis. Journal of Forest Economics 12(4): 251-277

Lindhjem H and Navrud S (2009) Asking for Individual or Household Willingness to Pay for Environmental Goods: Implication for aggregate welfare measures Environmental and Resource Economics 43(1): 11-29

Lindhjem H and Navrud S (Forthcoming) Using Internet in Stated Preference Surveys: A review and comparison of survey modes. International Review of Environmental and Resource Economics

MacDonald D H, Morrison M, Rose J and Boyle K (2010) Untangling Differences in Values from Internet and Mail Stated Preference Studies. World Congress of Environmental and Resource Economists, Montreal, Canada

MacMillan D, Hanley N and Lienhoop N (2006) Contingent valuation: Environmental polling or preference engine? Ecological Economics 60: 299-307

Maguire K B (2009) Does mode matter? A comparision of telephone, mail, and in-person treatments in contingent valuation surveys. Journal of Environmental Management 90: 3528-3533

Marta-Pedroso C, Freitas H and Domingos T (2007) Testing for the survey mode effect on contingent valuation data quality: a case study of web based versus in-person interviews. Ecological Economics 62: 388-398

McFadden D (1999) Rationality for economists? J. Risk Uncertain. 19(1-3): 73-105

Messonier M L, Bergstrom J C, Cornwell C M, Teasley R J and Cordell H K (2000) Survey Response-Related Biases in Contingent Valuation: Concepts, Remedies, and Empirical Application to Valuing Aquatic Plant Management. American Journal of Agricultural Economics 83: 438-450

Mitchell R C and Carson R T (1989) Using Surveys to Value Public Goods: The Contingent Valuation Method. Resources for the Future, Washington DC

Nielsen J S (In press) Use of the Internet for willingness-to-pay surveys. A comparison of face-to-face and web-based interviews. Resource and Energy Economics

OECD (2010). OECD Broadband Portal. http://www.oecd.org/sti/ict/broadband

OECD (2010). OECD Factbook.

Olsen S B (2009) Choosing Between Internet and Mail Survey Modes for Choice Experiment Surveys Considering Non-Market Goods Environmental and Resource Economics 44(4): 591-610

Payne J W, Bettman J R and Schade D A (1999) Measuring Constructed Preferences: Towards a Building Code. Journal of Risk and Uncertainty 19(1-3): 243-270

Roger J L, Howard K I and Vessey J T (1993) Using significance tests to evaluate equivalence between two experimental groups. Psychological Bulletin 113(3): 553-565

Schulze W, McClelland G, Waldman D and Lazo J H (1996) Sources of bias in contingent valuation. In: D. J. Bjornstad and J. Kahn (ed), The contingent valuation of environmental resources: Methodological resources and research needs. Edward Elgar,

Schuman H (1996) The sensitivity of CV outcomes to CV survey methods. In: D. J. Bjornstad and J. Kahn (ed), The contingent valuation of environmental resources: Methodological issues and research needs. Edward Elgar,

Shogren J F (2005) Experimental methods and valuation. In: K.-G. Maler and J. R. Vincent (ed), Handbook of Environmental Economics. . Amsterdam: North Holland,

Stanton J (1998) An empirical assessment of data collection using the internet. Personnel Psychology 51: 709-725 Statistics Norway (SSB) (2010). The Internet Poll, 2nd Quarter 2009

Taylor P A, Nelson N M, Grandjean B D, Anatchkova B and Aadland D (2009). Mode effects and other potential biases in panel-based Internet surveys: Final report.

Thurston H W (2006) Non-market valuation on the internet. In: A. Alberini and J. Kahn (ed), Handbook on contingent valuation. Edward Elgar,

Tourangeau R, Rips L J and Rasinski K A (2000) The psychology of survey response. Cambridge University Press, Cambridge

van der Heide C M, van den Bergh J C J M, van Ierland E C and Nunes P A L D (2008) Economic valuation of habitat defragmentation: A study of the Veluwe, the Netherlands. Ecological Economics 67: 205-216