The Mean Squared Prediction Error Paradox: A summary

Pincheira, Pablo and Hardy, Nicolas (2020): The Mean Squared Prediction Error Paradox: A summary.

Abstract

This is a summary of the paper entitled : “The Mean Squared Prediction Error Paradox”. In that paper, we show that traditional comparisons of Mean Squared Prediction Error (MSPE) between two competing forecasts may be highly controversial. This is so because when some specific conditions of efficiency are not met, the forecast displaying the lowest MSPE will also display the lowest correlation with the target variable. Given that violations of efficiency are usual in the forecasting literature, this opposite behavior in terms of accuracy and correlation with the target variable may be a fairly common empirical finding that we label here as "the MSPE Paradox." We characterize "Paradox zones" in terms of differences in correlation with the target variable and conduct some simple simulations to show that these zones may be non-empty sets. Finally, we illustrate the relevance of the Paradox with two empirical applications.

• Item Type: MPRA Paper
• Language: English
• Keywords: Mean Squared Prediction Error, Correlation, Forecasting, Time Series, Random Walk.
• Subjects: C - Mathematical and Quantitative Methods > C0 - General; C - Mathematical and Quantitative Methods > C0 - General > C00 - General; C - Mathematical and Quantitative Methods > C0 - General > C01 - Econometrics; C - Mathematical and Quantitative Methods > C0 - General > C02 - Mathematical Methods; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C21 - Cross-Sectional Models ; Spatial Models ; Treatment Effect Models ; Quantile Regressions; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C22 - Time-Series Models ; Dynamic Quantile Regressions ; Dynamic Treatment Effect Models ; Diffusion Processes; C - Mathematical and Quantitative Methods > C4 - Econometric and Statistical Methods: Special Topics; C - Mathematical and Quantitative Methods > C4 - Econometric and Statistical Methods: Special Topics > C41 - Duration Analysis ; Optimal Timing Strategies; C - Mathematical and Quantitative Methods > C4 - Econometric and Statistical Methods: Special Topics > C44 - Operations Research ; Statistical Decision Theory; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C51 - Model Construction and Estimation; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C52 - Model Evaluation, Validation, and Selection; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C53 - Forecasting and Prediction Methods ; Simulation Methods; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C54 - Quantitative Policy Modeling; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C58 - Financial Econometrics; E - Macroeconomics and Monetary Economics > E0 - General; E - Macroeconomics and Monetary Economics > E3 - Prices, Business Fluctuations, and Cycles; E - Macroeconomics and Monetary Economics > E3 - Prices, Business Fluctuations, and Cycles > E37 - Forecasting and Simulation: Models and Applications; E - Macroeconomics and Monetary Economics > E5 - Monetary Policy, Central Banking, and the Supply of Money and Credit; E - Macroeconomics and Monetary Economics > E5 - Monetary Policy, Central Banking, and the Supply of Money and Credit > E58 - Central Banks and Their Policies; E - Macroeconomics and Monetary Economics > E6 - Macroeconomic Policy, Macroeconomic Aspects of Public Finance, and General Outlook; F - International Economics > F3 - International Finance; F - International Economics > F3 - International Finance > F31 - Foreign Exchange; F - International Economics > F3 - International Finance > F37 - International Finance Forecasting and Simulation: Models and Applications; F - International Economics > F4 - Macroeconomic Aspects of International Trade and Finance; F - International Economics > F4 - Macroeconomic Aspects of International Trade and Finance > F41 - Open Economy Macroeconomics; F - International Economics > F4 - Macroeconomic Aspects of International Trade and Finance > F44 - International Business Cycles; F - International Economics > F4 - Macroeconomic Aspects of International Trade and Finance > F47 - Forecasting and Simulation: Models and Applications; G - Financial Economics > G0 - General > G00 - General; G - Financial Economics > G1 - General Financial Markets; G - Financial Economics > G1 - General Financial Markets > G12 - Asset Pricing ; Trading Volume ; Bond Interest Rates; G - Financial Economics > G1 - General Financial Markets > G15 - International Financial Markets; G - Financial Economics > G1 - General Financial Markets > G17 - Financial Forecasting and Simulation

References

1. Chen, Y.-C., Rogoff, K. S., and Rossi, B. (2010). Can Exchange Rates Forecast Commodity Prices ? Quarterly Journal of Economics, 125(August), 1145–1194.

2. Chen, Y.-C., Rogoff, K. S., and Rossi, B. (2011). Predicting Agri-Commodity Prices: An Asset Pricing Approach, World Uncertainty and the Volatility of Commodity Markets, ed. B. Munier, IOS.

3. Clark, T. E., and West, K. D. (2006). Using out-of-sample mean squared prediction errors to test the martingale difference hypothesis. Journal of econometrics, 135(1-2), 155-186.

4. Clark, T. E., and West, K. D. (2007). Approximately normal tests for equal predictive accuracy in nested models. Journal of econometrics, 138(1), 291-311.

5. Diebold, F. X., and Mariano, Roberto, S. (1995). Comparing Predictive Accuracy. Journal of Business and Economic Statics, 13(3), 253–263.

6. Ferraro, D., Rogoff, K., and Rossi, B. (2015). Can oil prices forecast exchange rates? An empirical analysis of the relationship between commodity prices and exchange rates. Journal of International Money and Finance, 54, 116-141.

7. Giacomini, R., and White, H. (2006). Tests of conditional predictive ability. Econometrica, 74(6), 1545-1578.

8. Ince, O., and Molodtsova, T. (2017). Rationality and forecasting accuracy of exchange rate expectations: Evidence from survey-based forecasts. Journal of International Financial Markets, Institutions and Money, 47, 131-151.

9. Meese, R, and Rogoff, K. (1983). Empirical exchange rate models of the seventies: Do they fit out of sample? Journal of International Economics, 14, 3–24.

10. Meese, R., and Rogoff, K. (1988). Was it real? The exchange rate‐interest differential relation over the modern floating‐rate period. the Journal of Finance, 43(4), 933-948.

11. Mincer, J. A., and Zarnowitz, V. (1969). The evaluation of economic forecasts. In Economic forecasts and expectations: Analysis of forecasting behavior and performance (pp. 3-46). NBER.

12. Pincheira, P., and Hardy, N. (2019a). Forecasting base metal prices with the Chilean exchange rate. Resources Policy, 62(February), 256–281.

13. Pincheira, P., and Hardy, N. (2019b). Forecasting Aluminum Prices with Commodity Currencies Forecasting Aluminum Prices with Commodity Currencies. Available at SSRN 3511564.

14. Stock, J. H., and Watson, M. (2003). Forecasting output and inflation: The role of asset prices. Journal of Economic Literature, 41(3), 788-829.

15. Timmermann, A. (2008). Elusive return predictability. International Journal of Forecasting, 24(1), 1-18.

16. Welch, I., and Goyal, A. (2008). A comprehensive look at the empirical performance of equity premium prediction. The Review of Financial Studies, 21(4), 1455-1508.

17. West, K. D. (1996). Asymptotic Inference about Predictive Ability. Econometrica, 64(5), 1067.

18. West, K. D. (2006). Chapter 3 Forecast Evaluation. Handbook of Economic Forecasting, 1(05), 99–134.