Incorporating Risk and Uncertainty in Cost-Benefit Analysis

Salci, Sener and Jenkins, Glenn (2016): Incorporating Risk and Uncertainty in Cost-Benefit Analysis.

Abstract

Cost-Benefit Analysis (CBA) is a tool for assessing the welfare effects of changes in regulatory and investment interventions. While in many ways an effective approach, a significant drawback of CBA, however, is that it relies on estimates for variables that cannot be predicted with complete accuracy. As such, expected outcomes generated by CBA, such as financial and economic net present values (NPVs), incorporate a degree of risk and uncertainty. It is therefore critical that CBA is based on transparent assumptions about the nature of risk and uncertainty affecting key variables: CBA cannot contribute to rational decision-making unless the distribution of outcomes is clear, and the effect on forecast reliability understood. Real-world risk and uncertainty generate numerous ex-ante outcomes at the point of appraisal. Correctly assessing risk and uncertainty is therefore one of the most difficult challenges decision-makers face in applying the results of CBA. This report offers a systematic approach to the incorporation of risk and uncertainty in CBA. The primary objectives are to review the professional literature on risk and uncertainty; to provide a methodology for taking account of risk and uncertainty in CBA; and to suggest guidelines for the interpretation and application of CBA results in the decision-making process. The treatment of risk and uncertainty are clearly addressed in the CBA guidelines of most OECD countries, although approaches vary. The simplest procedures are based on sensitivity analysis, as applied to a deterministic base case. More comprehensive analysis is based on assumed probability distributions for the variables concerned. The CBA guidelines of multilateral financial institutions and a number of advanced economies (Australia, Canada, France, the UK, the US and the European Union) call for sensitivity analysis on a project-by-project basis, identifying specific long-term risks and uncertainties associated with the assumptions and values used in appraisal and evaluation. Still greater insight into the impact of risk and uncertainty on expected regulatory outcomes can be gained from a probabilistic modeling of variable distributions and their inter-dependencies. A Monte Carlo simulation is therefore recommended alongside sensitivity analysis, where data, time and budget permit.

• Item Type: MPRA Paper
• Language: English
• Keywords: risk analysis, cost-benefit analysis, energy, nonrenewable resources, environment, Canada
• Subjects: D - Microeconomics > D6 - Welfare Economics > D61 - Allocative Efficiency ; Cost-Benefit Analysis; D - Microeconomics > D8 - Information, Knowledge, and Uncertainty; D - Microeconomics > D8 - Information, Knowledge, and Uncertainty > D81 - Criteria for Decision-Making under Risk and Uncertainty; N - Economic History > N5 - Agriculture, Natural Resources, Environment, and Extractive Industries; N - Economic History > N5 - Agriculture, Natural Resources, Environment, and Extractive Industries > N52 - U.S. ; Canada: 1913-; O - Economic Development, Innovation, Technological Change, and Growth > O1 - Economic Development > O13 - Agriculture ; Natural Resources ; Energy ; Environment ; Other Primary Products; Q - Agricultural and Natural Resource Economics ; Environmental and Ecological Economics > Q5 - Environmental Economics > Q51 - Valuation of Environmental Effects; Q - Agricultural and Natural Resource Economics ; Environmental and Ecological Economics > Q5 - Environmental Economics > Q53 - Air Pollution ; Water Pollution ; Noise ; Hazardous Waste ; Solid Waste ; Recycling; Q - Agricultural and Natural Resource Economics ; Environmental and Ecological Economics > Q5 - Environmental Economics > Q58 - Government Policy

References

Asche, F., Gjølberg, O. and Volker, T. 2003. Price relationships in the petroleum market: an analysis of crude oil and refined product prices. Energy Economics, 25, 289-301.

Asian Development Bank. 2002. Handbook for Integrating Risk Analysis in the Economic Analysis of Projects, Manila, Philippines http://www.adb.org/sites/default/files/institutional-document/32758/files/integrating-risk-analysis.pdf (last accessed on 2016-04-15)

Australia, Handbook of Cost-Benefit Analysis. 2006 (latest version). Department of Finance and Administration, Chapter 6: Allowing Risk and Uncertainty, p. 69-79 & Appendix IV: Full Risk Analysis, p.141-144. http://www.finance.gov.au/sites/default/files/Handbook_of_CB_analysis.pdf (last accessed on 2016-04-15)

Australia, Economic Evaluation for Business Cases. 2013. Technical Guidelines, the Secretary Department of Treasury and Finance, Section 8: Assessing Risk and Uncertainty, p.33-36

Australia, Project Assessment Framework, Cost-Benefit Analysis. 2015. The State of Queensland (Queensland Treasury), Appendix B1 – Risk Assessment, p.25, Appendix B4 – Sensitivity analysis, p.27-28 https://www.treasury.qld.gov.au/publications-resources/project-assessment-framework/paf-cost-benefit-analysis.pdf (last accessed on 2016-04-15)

Boardman, E.A., Greenberg, D.H., Vining, A.R., and Weimer, D.L. 2006. Dealing with Uncertainty’ in Cost-Benefit Analysis – Concepts and Practice, 3rd Ed., Pearson Prentice Hall, New Jersey, pp. 165-199.

Bock, K., and Truck, S. 2011.Assessing Uncertainty and Risk in Public Sector Investment Projects, Technology and Investment, 2: 105-123.

Borenstein, S., Cameron, C., and Gilbert, R. 1997. Do Gasoline Prices Respond Asymmetrically to Crude Oil Prices? Quarterly Journal of Economics, 112, 305-39.

Canada, Canadian Cost-Benefit Analysis Guide: Regulatory Proposals. 2007. Treasury Board of Canada Secretariat, Canada, 2007, p.4-7; and p. 22-24. http://www.tbs-sct.gc.ca/hgw-cgf/finances/rgs-erdg/wwad-cqnf/col/analys/analys-eng.pdf (last accessed on 2016-04-15)

Canada, Canada Gazette, July 20, 2011, ARCHIVED — Regulations Amending the Renewable Fuels Regulations, 145(15). http://canadagazette.gc.ca/rp-pr/p2/2011/2011-07-20/html/sor-dors143-eng.html (last accessed on 2016-04-15)

Canada, Canadian Regulatory Cost-Benefit Analysis Guide. 2007. Treasury Board of Canada, Secretariat in collaboration with Queen’s University and the Cost-Benefit Advisory Committee http://www.queensjdiexec.org/publications/qed_dp_168.pdf (last accessed on 2016-04-15)

Canada, Treasury Board of Canada. 2007. Cost-Benefit Analysis Case Study on Regulations to Lower the Level of Sulphur in Gasoline, in collaboration with Queen’s University and the Cost-Benefit Advisory Committee, Authors: Jenkins, G.P, Kuo, C.Y. and Ozbafli, A.

Canada, Natural Resources Canada. 2011. Canadian Crude Oil, Natural Gas and Petroleum Products: Review of 2009 & Outlook to 2030, Petroleum Research Brand, Energy Sector http://www.nrcan.gc.ca/sites/www.nrcan.gc.ca/files/energy/pdf/eneene/sources/crubru/revrev/pdf/revrev-09-eng.pdf (last accessed on 2016-04-15)

Canada, Natural Resources Canada, 2014-2015, Energy Markets Fact Book 2014–2015 http://www.nrcan.gc.ca/sites/www.nrcan.gc.ca/files/energy/files/pdf/2014/14-0173EnergyMarketFacts_e.pdf (last accessed on 2016-04-15)

Canada, Environment and Climate Change Canada. April 2016. Canada’s Emissions Trends, 1990-2014, https://www.ec.gc.ca/indicateurs-indicators/F60DB708-6243-4A71-896B-6C7FB5CC7D01/GHGEmissions_EN.pdf (last accessed on 2016-04-25)

Clarke, R and Low, A. 1993, Risk Analysis in Project Planning: A Simple Spreadsheet Application Using Monte-Carlo Techniques, Project Appraisal, 8, 3, pp. 141-146

Clarke, H. 2014 “Evaluating Infrastructure Projects Under Risk and Uncertainty: A Checklist of Issuers”, Australian Economic Review, 47, 1, 147-156.

Coyle, W. 2013. The Future of Biodiesel - A Global Perspective. USDA.gov: http://www.ers.usda.gov/amber-waves/2007-november/the-future-of-biofuels-a-global-perspective.aspx#.Vw09x_krLIU (last accessed on 2016-04-15)

European Union, Guide to Cost-Benefit Analysis of Investment Projects Economic appraisal tool for Cohesion Policy 2014-2020. 2014. European Commission. Directorate-General for Regional and Urban policy http://ec.europa.eu/regional_policy/sources/docgener/studies/pdf/cba_guide.pdf (last accessed on 2016-04-15)

Fao, R., Howard, M. 2006. Use of Monte Carlo Simulation for the Public Sector: An Evidence Based Approach to Scenario Planning, International Journal of Market Research, 48, 27-48

Flyvberg, B. 2009. Survival of the Unfittest: Why the Worst Infrastructure Gets Built, Oxford Economic Review, 25, pp. 344-367

France, Guide de l’analyse coûts-avantages des projets d’investissement, 2003. 2.7.4: Analyse de probabilité des risques, p. 41-45, Annexe D: Analyse de sensibilité et des risques, p.123-125 http://www2.toulouse.inra.fr/lerna/treich/ACB03.pdf(last accessed on 2016-04-15)

France, Gollier, C., Baumstark, L., Fery, P. Dominique A. and Raynard, C. 2011. Le calcul du risque dans les investissements publics http://www.ladocumentationfrancaise.fr/var/storage/rapports-publics/114000605.pdf (Last accessed on 2016-04-15)

Greenstone, M., Kopits, E. and Wolverton, A. 2011. Estimating the social cost of carbon for use in U.S. federal rulemakings: A summary and interpretation. NBER Working Paper 16913, National Bureau of Economic Research, Cambridge, MA. http://web.mit.edu/ceepr/www/publications/workingpapers/2011-006.pdf (last accessed on 2016-04-15)

Hillson, D. and Murray-Webster, R. 2004. Understanding and managing risk attitude. Available at: http://www.kent.ac.uk/scarr/events/finalpapers/Hillson%20%2B%20MurrayWebster.pdf

Jenkins, G.P., 1997, Project Analysis and the World Bank, American Economic Review, 87, 2, pp. 38-42

Jenkins, G.P., Kuo, C-Y., and Harberger, A.C. 2011. Cost-Benefit Analysis for Investment Decisions: Chapter 6: Dealing with Uncertainty and Risk in Investment Appraisal, available under Development Discussion Paper: 2011-6, John Deutsch International, Queen’s University, Canada http://www.queensjdiexec.org/publications/qed_dp_199.pdf (last accessed on 2016-04-15)

Johnson, L.T. and Hope, C. 2012. The social cost of carbon in U.S. regulatory impact analyses: an introduction and critique. J Environ Stud Scince.

Kwak, Y.H., and Ingall, L. 2007. Exploring Monte Carlo Simulation Applications for Project Management, Risk Management, 44-57

Kriger, D., Shie, S., and Naylor, S. 2006. Estimating Toll Road Demand and Revenue: A Synthesis of Highway Practice, Transportation Research Board, NCHRP, Washington DC, US

Lemp, J.D., Kockelman, K.M. 2009. Understanding and Accommodating Risk and Uncertainty in Toll Road Projects: A Review of Literature, Transport Research Record, 2132, 106-112

Li, R. 2010. The Relationships among Petroleum Prices. International Conference on Applied Economics – ICOAE 2010, 459-463

Mietzner, D., and Reger, G. 2005. Advantages and disadvantages of scenario approaches for strategic foresight, Int. J. Technology Intelligence and Planning, 1, 220-239

New Zealand, The Treasury, Guide to Social Cost Benefit Analysis. 2015. Issues in Cost Benefit Analysis, p.50-51 & Appendix 1: Monte Carlo Simulation, p.60-61 http://www.treasury.govt.nz/publications/guidance/planning/costbenefitanalysis/guide/cba-guide-jul15.pdf (last accessed on 2016-04-15)

Pokrivcak, J., and Rajcaniova, M. 2011. Crude oil price variability and its impact on ethanol prices. Agric Econ - Czech, 8, 394-403.

Sanderson, J. 2012. Risk, Uncertainty and Governance in Megaprojects: A Critical Discussion of Alternative Explanations, International Journal of Project Management, 30, 432-443

Savvides, S. 1994. Risk Analysis in Investment Appraisal, Project Appraisal, 9, 1-30

Tareen, I.Y., Wetzstein, M.E., Duffield, J.A. 2000. Biodiesel as a substitute for petroleum diesel in a stochastic environment. Journal of Agricultural and Applied Economics, 32, 373-381.

United Kingdom, the Green Book, Appraisal and Evaluation in Central Government. 2015. HM Treasury of the United Kingdom, Treasury Guidance, Chapter 5: Appraising the Options, p. 17-34 & Annex 4 Risk and uncertainty, p.79-88 https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/220541/green_book_complete.pdf (last accessed on 2016-04-15).

United States of America, Environmental Protection Agency, US EPA. May 2014. Guidelines for Preparing Economic Analyses https://www.epa.gov/sites/production/files/2014-11/documents/montecar.pdf (last accessed on 2016-04-15). http://yosemite.epa.gov/ee/epa/eerm.nsf/vwAN/EE-0568-50.pdf/$file/EE-0568-50.pdf

United States of America, Environmental Protection Agency, US EPA, 1997 Guiding principles for Monte Carlo analysis. Risk Assessment Forum, Washington, DC. http://www.epa.gov/osa/basic-information-about-risk-assessment-guidelines-development

United States of America, White House, January 11 1996. Economic Analysis of Federal Regulations, Executive Order 12866 https://www.whitehouse.gov/omb/inforeg_riaguide (last accessed on 2016-04-15).

United States of America, Environmental Protection Agency, US EPA. 2010. Technical Support Document: Social Cost of Carbon for Regulatory Impact Analysis under Executive Order 12866 Interagency Working Group on Social Cost of Carbon 47 https://www3.epa.gov/otaq/climate/regulations/scc-tsd.pdf (last accessed on 2016-04-15)

Van Gronendal, W.J.H., and Kleijnen, J.P.C. 2002. Deterministic versus Stochastic Sensitivity Analysis in Investment Decisions: An Environmental Case Study, European Journal of Operational Research, 141, 8-20

Zwikael, O., and Ahn, M. 2011. The effectiveness of risk management: An analysis of project risk management planning across industries and countries. Risk Analysis, 31, 25-37