Correlation Based Tests of Predictability

Pincheira, Pablo and Hardy, Nicolas (2022): Correlation Based Tests of Predictability.

Abstract

In this paper, we propose a correlation-based test for the evaluation of two competing forecasts. Under the null hypothesis of equal correlations with the target variable, we derive the asymptotic distribution of our test using the Delta method. This null hypothesis is not necessarily equivalent to the null of equal Mean Squared Prediction Errors (MSPE). Specifically, it might be the case that the forecast displaying the lowest MSPE also exhibits the lowest correlation with the target variable: this is known as "The MSPE paradox" (Pincheira and Hardy; 2021). In this sense, our approach should be seen as complementary to traditional tests of equality in MSPE. Monte Carlo simulations indicate that our test has good size and power. Finally, we illustrate the use of our test in an empirical exercise in which we compare two different inflation forecasts for a sample of OECD economies. We find more rejections of the null of equal correlations than rejections of the null of equality in MSPE.

• Item Type: MPRA Paper
• Language: English
• Keywords: Forecasting, time-series, out-of-sample evaluation, mean squared prediction error, correlations.
• Subjects: C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C52 - Model Evaluation, Validation, and Selection; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C53 - Forecasting and Prediction Methods ; Simulation Methods; E - Macroeconomics and Monetary Economics > E3 - Prices, Business Fluctuations, and Cycles > E31 - Price Level ; Inflation ; Deflation; E - Macroeconomics and Monetary Economics > E3 - Prices, Business Fluctuations, and Cycles > E37 - Forecasting and Simulation: Models and Applications; F - International Economics > F3 - International Finance > F37 - International Finance Forecasting and Simulation: Models and Applications; G - Financial Economics > G1 - General Financial Markets > G17 - Financial Forecasting and Simulation

References

1. Akoglu, H. (2018). User's guide to correlation coefficients. Turkish journal of emergency medicine, 18(3), 91-93.

2. Andrews, D. (1991). Heteroskedasticity and autocorrelation consistent covariance matrix estimation. Econometrica: Journal of the Econometric Society, 817-858.

3. Ang, A., Bekaert, G., and Wei, M. (2007) Do macro variables, asset markets, or surveys forecast inflation better? Journal of Monetary Economics, Volume 54, issue 4, 1163-1212.

4. Bermingham, C. (2007). How Useful is Core Inflation for Forecasting Headline Inflation?. Economic & Social Review, 38(3).

5. Bentancor, A., and Pincheira, P. (2010) Predicción de errores de proyección de inflación en Chile. El Trimestre Económico, Vol (0) 305, 129-154. (In Spanish).

6. Bullard, J. (2011). Measuring inflation: the core is rotten. Federal Reserve Bank of St. Louis Review, 93(4), 223-233.

7. Busetti, F., and Marcucci, J. (2013). Comparing forecast accuracy: a Monte Carlo investigation. International Journal of Forecasting, 29(1), 13-27.

8. Ciccarelli, M., and Mojon, B. (2010). Global inflation. The Review of Economics and Statistics, 92(3), 524-535.

9. Clark, T., and McCracken, M. (2001). Tests of equal forecast accuracy and encompassing for nested models. Journal of Econometrics, 105(1), 85-110.

10. Clark, T., and McCracken, M. (2005): "Evaluating Direct Multistep Forecasts". Econometric Reviews 24(4): 369-404.

11. Clark, T., and McCracken, M. (2013). Advances in forecast evaluation. Handbook of Economic Forecasting, 2, 1107-1201.

12. Clark, T., and West, K. (2006): "Using Out-of-Sample Mean Squared Prediction Errors to Test the Martingale Difference Hypothesis". Journal of Econometrics 135: 155-186.

13. Clark, T., and West, K. (2007). Approximately normal tests for equal predictive accuracy in nested models. Journal of Econometrics, 138(1), 291-311.

14. Diebold, F., and Mariano, R. (1995): "Comparing Predictive Accuracy". Journal of Business & Economic Statistics 13: 253-263.

15. Duncan, R., and Martínez-García, E. (2015). Forecasting local inflation with global inflation: When economic theory meets the facts. Globalization and Monetary Policy Institute Working Paper, (235).

16. Elliott, G., and Timmermann, A. (2013). Handbook of Economic Forecasting. Elsevier.

17. Freeman, D. G. (1998). Do core inflation measures help forecast inflation?. Economics Letters, 58(2), 143-147.

18. Giacomini, R., and White,H. (2006): "Tests of Conditional Predictive Ability". Econometrica 74(6): 1545-1578.

19. Granger, C., and Machina, J. (2006). Forecasting and decision theory. Handbook of Economic Forecasting, 1, 81-98.

20. Granger, C. (1999). Empirical modeling in economics: Specification and evaluation. Cambridge University Press.

21. Granger, C., and Newbold, P. (1986). Forecasting Economic Time Series. Elsevier Monographs.

22. Hakkio, C. (2009). Global inflation dynamics. Federal Reserve Bank of Kansas City Working Paper, (09-01).

23. Hansen, B. (2019). Econometrics. University of Wisconsin. www.ssc.wisc.edu/∼bhansen.

24. Ince, O., and Molodtsova, T. (2017). Rationality and forecasting accuracy of exchange rate expectations: Evidence from survey-based forecasts. Journal of International Financial Markets, Institutions and Money, 47, 131-151.

25. Joutz, F. and Stekler, H., (2000). An Evaluation of the Prediction of the Federal Reserve. International Journal of Forecasting. 16: 17–38.

26. Kabukçuoğlu, A., and Martínez-García, E. (2018). Inflation as a global phenomenon—Some implications for inflation modeling and forecasting. Journal of Economic Dynamics and Control, 87, 46-73.

27. Le Bihan, H., and Sédillot, F. (2000). Do core inflation measures help forecast inflation?: Out-of-sample evidence from French data. Economics Letters, 69(3), 261-266.

28. McCracken M. (2007): "Asymptotics for Out of Sample Tests of Granger Causality". Journal of Econometrics 140: 719-752.

29. Medel, C., Pedersen, M., and Pincheira, P. (2016). The elusive predictive ability of global inflation. International Finance, 19(2), 120-146.

30. Mincer, J. A., and Zarnowitz, V. (1969). The evaluation of economic forecasts. In Economic forecasts and expectations: Analysis of forecasting behavior and performance (pp. 3-46). NBER.

31. Morales‐Arias, L., and Moura, G. V. (2013). A conditionally heteroskedastic global inflation model. Journal of Economic Studies.

32. Newey, W., and West, K. (1987). Hypothesis testing with efficient method of moments estimation. International Economic Review, 777-787.

33. Newey, W., and West, K. (1994). Automatic lag selection in covariance matrix estimation. The Review of Economic Studies, 61(4), 631-653.

34. Nordhaus, W. (1987). Forecasting efficiency: concepts and applications. The Review of Economics and Statistics, Vol LXIX, N°4.

35. Patton, A., and Timmermann, A. (2012). Forecast Rationality Tests Based on Multi-Horizon Bounds ". Journal of Business and Economic Statistics 30(1): 1–17.

36. Pincheira, P. (2022). A Power Booster Factor for Out-of-Sample Tests of Predictability. Economia, forthcoming.

37. Pincheira, P. (2010). A Real time evaluation of the Central Bank of Chile GDP growth forecasts. Money Affairs, CEMLA, vol. 0(1), 37-73.

38. Pincheira, P. (2012). A joint test of superior predictive ability for Chilean inflation forecasts. Journal Economía Chilena (The Chilean Economy), Central Bank of Chile, vol. 15(3), 04-39, (In Spanish).

39. Pincheira, P., and Álvarez, R. (2009). Evaluation of short run inflation forecasts and forecasters in Chile," Money Affairs, CEMLA, vol. 0(2), 159-180.

40. Pincheira, P. and Fernández, N. (2011). Corrección de algunos errores sistemáticos de predicción de inflación. Monetaria, CEMLA, vol. 0(1), 37-61. (In Spanish).

41. Pincheira, P., and Gatty, A. (2016). Forecasting Chilean inflation with international factors. Empirical Economics, 51(3), 981-1010.

42. Pincheira, P., and Hardy, N. (2021). The Mean Squared Prediction Error Paradox (No. 107403). University Library of Munich, Germany.

43. Pincheira, P., Hardy, N., and Muñoz, F. (2021). "Go Wild for a While!": A New Test for Forecast Evaluation in Nested Models. Mathematics, 9(18), 2254.

44. Pincheira, P., Selaive, J., and Nolazco, J. (2019). Forecasting inflation in Latin America with core measures. International Journal of Forecasting, 35(3), 1060-1071.

45. Robalo, C., Duarte, P., and Morais, L. (2003). "Evaluating core inflation indicators." Economic Modelling 20, 765-775.

46. Samuel, M., and Okey, L. (2015). The relevance and significance of correlation in social science research. International Journal of Sociology and Anthropology Research, 1(3), 22-28.

47. Song, L. (2005). Do underlying measures of inflation outperform headline rates? Evidence from Australian data. Applied Economics, 37(3), 339-345.

48. West K. (1996):" Asymptotic Inference About Predictive Ability". Econometrica 64: 1067-1084.

49. West K. (2006): "Forecast Evaluation" Handbook of Economic Forecasting, Elsevier.

50. White, H (2000). Asymptotic theory for econometricians. Academic press.