The Evolution of Inefficiency in USDA’s Forecasts of U.S. and World Soybean Markets

MacDonald, Stephen and Ash, Mark and Cooke, Bryce (2017): The Evolution of Inefficiency in USDA’s Forecasts of U.S. and World Soybean Markets.

Abstract

We derive a set of stylized facts about USDA’s soybean supply and demand forecasts and draw implications from these results for efforts to improve the accuracy of these forecasts. USDA’s short run soybean supply and demand forecasts are inefficient, with several key variables significantly biased throughout much of the annual forecast cycle. Bias and other characteristics have varied significantly over recent decades, and evaluation efforts intended to guide forecast improvement need to focus on very recent sample years and seasonally disaggregated data. USDA should expand the information set used in its forecasting of U.S. soybean yields, take note of its mid-season downward bias in its Brazilian production forecasts, and shift the weight of its early-season focus for China’s soybean consumption estimates towards forecasts of growth rates and reduce the weight it applies to forecasting consumption and trade by China in terms of levels. During 2004-2015, downward-biased U.S. production forecasts and upward-biased foreign excess supply forecasts resulted in downward-biased U.S. export forecasts throughout much of USDA’s annual forecasting cycle. Twenty years earlier, this downward bias was confined to the end of the forecasting cycle, whereas U.S. soybean ending stock forecasts have been upward biased for decades across much of the forecasting cycle. The forecasts for U.S. soybean exports are also characterized by smoothing, with strong correlation between month-to-month forecast revisions towards the later months of USDA’s annual forecasting cycle.

• Item Type: MPRA Paper
• Language: English
• Keywords: Forecast evaluation, overlapping forecasts, bias, smoothing, agriculture, soybeans, China, United States, Brazil, Argentina, USDA, WASDE
• Subjects: C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C53 - Forecasting and Prediction Methods ; Simulation Methods; Q - Agricultural and Natural Resource Economics ; Environmental and Ecological Economics > Q0 - General > Q02 - Commodity Markets; Q - Agricultural and Natural Resource Economics ; Environmental and Ecological Economics > Q1 - Agriculture > Q11 - Aggregate Supply and Demand Analysis ; Prices; Q - Agricultural and Natural Resource Economics ; Environmental and Ecological Economics > Q1 - Agriculture > Q17 - Agriculture in International Trade

References

Adjemian, M., (2012). “Quantifying the WASDE Announcement Effect,” American Journal of Agricultural Economics, 94 (1): 238-256.

Bailey, D.V., and B.W. Brorsen (1998). “Trends in the Accuracy of USDA Production Forecasts for Beef and Pork.” Journal of Agricultural and Resource Economics 23: 515–26.

Baur, R.F., and P.F. Orazem (1994). “The Rationality and Price Effects of USDA Forecasts of Oranges.” Journal of Finance 49: 681-696.

Botto, A, Isengildina, O, Irwin, S, and D. Good, (2006). “Accuracy Trends and Sources of Forecast Errors in WASDE Balance Sheet Categories for Corn and Soybeans,” Selected Paper prepared for presentation at the American Agricultural Economics Association Annual Meeting, Long Beach, California, July 23-26.

Cheung , Y.-W. , Chinn , M. , Garcia Pascual , A. (2005). “Empirical exchange rate models of the nineties: Are any fit to survive?”, Journal of International Money and Finance, 24:1150 – 1175.

Clough, Malcolm (1951). “Changes in Corn Acreage and Production After the Early Indications,” Agricultural Economics Research, No. 4 1951.

Coibion O., and Y. Gordonichenko (2010). “Information Rigidity and the Expectations Formation Process: A Simple Framework and New Facts.” Working Paper, No. 16537, National Bureau of Economic Research.

Colling, P.L., and S.H. Irwin (1990). "The Reaction of Live Hog Futures Prices to USDA Hogs and Pigs Reports." American Journal of Agricultural Economics 71:84-94.

Davies, A., and K.Lahiri (1995). "A New Framework for Testing Rationality and Measuring Aggregate Shocks Using Panel Data." Journal of Econometrics 68:205-227.

Diebold, F.X., and J.A. Lopez (1998). “Forecasting Evaluation and Combination.” In Handbook of Statistics 14: Statistical Methods in Finance, eds., G.S. Maddala and C.R. Rao, pp.241-68. Amsterdam: North-Holland.

Effland, Anne (1999). “When the 20th Century was the Future: USDA’s ‘New’ Outlook Program and the ‘Future’ of Public Forecasting,” Papers and Proceedings: The 10th Federal Forecasters Conference, ed. Debra Gerald, U.S. Department of Education, National Center for Education Statistics.

Egelkraut, Thorsten, Garcia, Philip, Irwin, Scott, and Darrel Good (2003). “An Evaluation of Crop Forecast Accuracy for Corn and Soybeans: USDA and Private Information Agencies,” Journal of Agricultural and Applied Economics, 35(1): 79-95.

Fortenbery, T. R., and D.A. Sumner (1993). “The Effects of USDA Reports in Futures and Options Markets.” Journal of Futures Markets 13: 157-173.

General Accounting Office (1988). USDA’s Commodity Program: The Accuracy of Budget Forecasts. GAO.PEMD-88-8

General Accounting Office (1991). USDA’s Commodity Forecasts: Inaccuracies Found May Lead to Underestimates of Budget Outlays. GAO/PEMD-91-24.

Good, Darrel, and Scott Irwin (2006). Understanding USDA Corn and Soybean Production Forecasts: Methods, Performance and Market Impacts over 1970 – 2005, AgMAS Project Research Report 2006-01.

Granger, C. W. J. (1980). Testing for causality: A personal viewpoint. Journal of Economic Dynamics and Control 2, 329-352.

Granger C.W.J. and Paul Newbold (1986). Forecasting Economic Time Series (Second Edition), New York: Harcourt Brace Jovanovich.

Gunnelson, G., W. Dobson, and S. Pamperin (1972). “Analysis of the Accuracy of USDA Crop Forecasts.” American Journal of Agricultural Economics, 54(1972): 639-645.

Holden K., and D.A. Peel (1990), “On Testing for Unbiasedness and Efficiency of Forecasts,” The Manchester School 58(2): 120-7.

Irwin, Scott, Sanders, Dwight, and Darrel Good (2014) “Evaluation of Selected USDA WAOB and NASS Forecasts and Estimates in Corn and Soybeans." Marketing and Outlook Research Report 2014-01, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign.

Isengildina, Olga, Irwin, Scott, and Darrel Good (2006). “Are Revisions to USDA Crop Production Forecasts Smoothed?” American Journal of Agricultural Economics, 88(4): 1091-1104.

Isengildina-Massa, Olga, Irwin, Scott, and Darrel Good (2013). “Do Big Crops Get Bigger and Small Crops Get Smaller? Further Evidence on Smoothing in U.S. Department of Agriculture Forecasts,” Journal of Agricultural and Applied Economics, 45(1): 95-107.

Isengildina-Massa, O., Irwin, S., Good, D., and J. Gomez, (2008). “The Impact of Situation and Outlook Information in Corn and Soybean Futures Markets: Evidence from WASDE Reports,” Journal Agricultural and Applied Economics, 40:89-103.

Isengildina-Massa, Olga, Karali, Berna and Scott H. Irwin (2013). “When do the USDA forecasters make mistakes?,” Applied Economics, 45:36, 5086-5103

Issengildina-Massa, O., MacDonald, S., and R. Xie, (2012). “A Comprehensive Evaluation of USDA Cotton Forecasts,” Journal of Agricultural and Resource Economics, 37(1): 98-113.

Lewis, K. and M. Manfredo (2012). “An Evaluation of the USDA Sugar Production and Consumption Forecasts.” Paper presented at the NCCC-134 Conference on Applied Commodity Price Analysis, Forecasting, and Market Risk Management, St. Louis, Missouri, April 16-17, 2012.

MacDonald, S, (1992). The Accuracy of USDA’s Export Forecasts, Staff Report No. AGES 9224, U.S. Department of Agriculture, Economic Research Service.

MacDonald, S. (1999). “A Preliminary Evaluation of USDA’s Export Forecasts,” In Papers and Proceedings: 10th Federal Forecasters Conference, 1999, ed. Federal Forecasters Governing Board, pages 303-311

MacDonald, S. (2002). “Rational Commodity Forecasts: Improving USDA’s Cotton Analysis,” In Papers and Proceedings: 12th Federal Forecasters Conference, 2002, ed. Debra Gerald, U.S. Department of Education, National Center for Educational Statistics, pages 355-360.

MacDonald, S. (2005). “A Comparison of USDA’s Agricultural Export Forecasts with ARIMA-based Forecasts.” In Papers and Proceedings: 14th Federal Forecasters Conference 2005, ed. Federal Forecasters Governing Board, pages 151-165.

MacDonald, S. and Olga Isengildina-Massa (2012a). “Information Rigidity and Correcting Inefficiency in USDA’s Commodity Forecasts,” Selected Poster prepared for presentation at the Agricultural & Applied Economics Association’s Annual Meeting, Seattle, Washington, August 12-14.

MacDonald, S. and Olga Isengildina-Massa (2012b). “Adjustment Strategies for Forecast Smoothing: A Soybean Production Forecasts Case Study,” Presentation at Federal Forecasters Conference, Washington, DC, September 27, 2012.

Meese, Richard, and Kenneth Rogoff (1983). “Empirical Exchange Rate Models of the Seventies: Do they fit out of sample?”, Journal of International Economics, 14:3-24.

Mincer, J, and V. Zarnowitz (1969). "The Evaluation of Economic Forecasts," In: Economic Forecasts and Expectations: Analysis of Forecasting Behavior and Performance, National Bureau of Economic Research.

Nordhaus, W., (1987). “Forecast Efficiency: Concepts and Applications,” Review of Economics and Statistics, 69(4): 667-674.

Organization for Economic Cooperation and Development and Food and Agriculture Organization of the United Nations, (2016). OECD-FAO Agricultural Outlook 2016-2025.

Pons, J. (2000). “The Accuracy of IMF and OECD Forecasts for G7 Countries.” Journal of Forecasting 19: 53-63.

Sanders, D. R., and M. R. Manfredo (2002). “USDA Production Forecasts for Pork, Beef, and Broilers: An Evaluation.” Journal of Agricultural and Resource Economics 27:114-28.

Sanders, D. R., and M. R. Manfredo (2003). “USDA Livestock Price Forecasts: a Comprehensive Evaluation.” Journal of Agricultural and Resource Economics 28:316-334.

Sanders, D. R., and M. R. Manfredo (2008). “Multiple Horizons and Information in USDA Production Forecasts,” Agribusiness, 24(1):55–66.

Thompson, J.M. (1974). “Analysis of the Accuracy of USDA Hog Farrowings Statistics.” American Journal of Agricultural Economics, 34: 1213-1217.

Tversky, A. and D. Kehneman (1981). The Framing of Decisions and the Psychology of Choice, Science, 211: 453-458.

U.S. Department of Agriculture, (2004) Transcript, Release No. 0365.04. http://www.usda.gov/wps/portal/usda/usdamobile?contentid=2004/08/0365.xml

U.S. Department of Agriculture, Foreign Agricultural Service, (2002-2017). Production, Supply, and Distribution, Downloadable Datasets.

U.S. Department of Agriculture, Office of Chief Economist, (1998-2016). U.S. Agricultural Baseline Projections. Staff Report WAOB 98-1 through WAOB 2005-1, OCE-2005-1 through OCE-2016-1.

U.S. Department of Agriculture, Economic Research Service, (2016). Farm Income and Wealth Statistics. https://data.ers.usda.gov/reports.aspx?ID=53581

U.S. Department of Agriculture, Economic Research Service (1993-2002). Production, Supply and Distribution View, Stock #93002E.

U.S. Department of Agriculture, Wheat, Feed Grains, and Oilseeds Interagency Commodity Estimates Committees (1998-2016). “Grains and Oilseeds Outlook,” Presentation at USDA Agricultural Outlook Forum.

U.S. Department of Agriculture, World Agricultural Outlook Board, (1998-2016), World Agricultural Supply and Demand Estimates, WASDE-338 through WASDE-561.

U.S. Department of Agriculture, World Agricultural Outlook Board, (2016), World Agricultural Supply and Demand Estimates, WADE-553.

Vogel, F.A., and G.A. Bange (1999). “Understanding USDA Crop Forecasts.” Miscellaneous Publication No. 1554, US Department of Agriculture, National Agricultural Statistics Service and Office of the Chief Economist, World Agricultural Outlook Board.

Williams, J.C., and B. D. Wright (1991). Storage and Commodity Markets, Cambridge University Press, Cambridge, U.K.

Xao, Jinzhi, Lence, Sergio, and Chad Hart (2014). “USDA And Private Analysts' Forecasts Of Ending Stocks: How Good Are They?,” Paper provided by Agricultural and Applied Economics Association in its series 2014 Annual Meeting, July 27-29, 2014, Minneapolis, Minnesota.