Are some forecasters really better than others?

D'Agostino, Antonello and McQuinn, Kieran and Whelan, Karl (2011): Are some forecasters really better than others?

Abstract

In any dataset with individual forecasts of economic variables, some forecasters will perform better than others. However, it is possible that these ex post differences reflect sampling variation and thus overstate the ex ante differences between forecasters. In this paper, we present a simple test of the null hypothesis that all forecasters in the US Survey of Professional Forecasters have equal ability. We construct a test statistic that reflects both the relative and absolute performance of the forecaster and use bootstrap techniques to compare the empirical results with the equivalents obtained under the null hypothesis of equal forecaster ability. Results suggest little support for the idea that the best forecasters are actually innately better than others, though there is evidence that a relatively small group of forecasters perform very poorly.

• Item Type: MPRA Paper
• Language: English
• Keywords: Forecasting, Bootstrap
• Subjects: C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C53 - Forecasting and Prediction Methods ; Simulation Methods; E - Macroeconomics and Monetary Economics > E2 - Consumption, Saving, Production, Investment, Labor Markets, and Informal Economy > E27 - Forecasting and Simulation: Models and Applications; E - Macroeconomics and Monetary Economics > E3 - Prices, Business Fluctuations, and Cycles > E37 - Forecasting and Simulation: Models and Applications

References

Bonham, Carl and Richard Cohen (2001), “To Aggregate, Pool, or Neither: Testing theRational Expectations Hypothesis Using Survey Data,” Journal of Business and Economic Statistics, 19, 278291.

Batchelor, Roy A. (1990), “All Forecasters are Equal.” Journal of Business and Economic Statistics 8, 143 - 144.

Batchelor, Roy A. and Pami Dua (1990), “Product Differentiation in the Economic Forecasting Industry.” International Journal of Forecasting 6, 311-316.

Christensen H. Jens, Francis X. Diebold, Georg H. Strasser and Glenn D. Rudebusch (2008). “Multivariate Comparison of Predictive Accuracy”, working paper available at http://www.econ.uconn.edu/Seminar%20Series/strasser08.pdf .

Cuthbertson, Keith, Dirk Nitzsche and Niall O’Sullivan (2008). “UK Mutual Fund Performance: Skill or Luck?” Journal of Empirical Finance, 15, 613-634.

D’Agostino, Antonello, Domenico Giannone, and Paolo Surico (2006), “(Un)Predictability and macroeconomic stability,” Working Paper Series 605, European Central Bank.

Diebold, Francis. X. and Mariano, Roberto (1995). “Comparing Predictive Accuracy,” Journal of Business and Economic Statistics, 13, 253263.

Fama, Eugene F and Kenneth French (2010). “Luck versus Skill in the Cross Section of Mutual Fund Returns,” forthcoming, Journal of Finance.

Grossman, Sanford and Joseph Stiglitz (1980). “On the Impossibility of Informationally Efficient Markets,” American Economic Review, 70, 393-408.

Keane, Michael and David Runkle (1990). “Testing the Rationality of Price Forecasts: New Evidence from Panel Data,” American Economic Review, 80, 714-735.

Kosowski, Robert, Allan Timmerman, Russ Wermers and Hall White (2006). “Can Mutual Fund “Stars” Really Pick Stocks? New Evidence from a Bootstrap Analysis,” Journal of Finance, 56, 2551-2595.

Laster, David, Paul Bennett and In Sun Geoum. “Rational Bias in Macroeconomic Forecasts”, Quarterly Journal of Economics, 114, 293-318.

Patton, Andrew and Alan Timmerman (2007). “Testing Forecast Optimality under Unknown Loss,” Journal of the American Statistical Association, 102, 1172-1184.

Romer, David and Christine Romer (2000) “Federal Reserve information and the behavior of interest rates”, American Economic Review 90, 429-457.

Stekler, Herman. (1987), “Who Forecasts Better?” Journal of Business and Economic Statistics 5, 155 - 158.

Stock, James and Mark Watson (2005). Has inflation become harder to forecast? Prepared for the conference “Quantitative Evidence on Price Determination”, Board of Governors of the Federal Reserve Board, September 29-30, Washington DC.

Stock, James and Mark Watson (2006). Why has U.S. inflation become harder to forecast? National Bureau of Economic Research (NBER) Working paper 12324.

Zarnowitz, Victor and Philip Bruan (1993). “Twenty Two Years of the NBER-ASA Quarterly Economic Outlook Surveys” in James Stock and Mark Watson (eds.) Business Cycle Indicators and Forecasting, University of Chicago Press.