"Go wild for a while!": A new asymptotically Normal test for forecast evaluation in nested models

Pincheira, Pablo and Hardy, Nicolás and Muñoz, Felipe (2021): "Go wild for a while!": A new asymptotically Normal test for forecast evaluation in nested models.

Abstract

In this paper we present a new asymptotically normal test for out-of-sample evaluation in nested models. Our approach is a simple modification of a traditional encompassing test that is commonly known as Clark and West test (CW). The key point of our strategy is to introduce an independent random variable that prevents the traditional CW test from becoming degenerate under the null hypothesis of equal predictive ability. Using the approach developed by West (1996), we show that in our test the impact of parameter estimation uncertainty vanishes asymptotically. Using a variety of Monte Carlo simulations in iterated multi-step-ahead forecasts we evaluate our test and CW in terms of size and power. These simulations reveal that our approach is reasonably well-sized even at long horizons when CW may present severe size distortions. In terms of power, results are mixed but CW has an edge over our approach. Finally, we illustrate the use of our test with an empirical application in the context of the commodity currencies literature.

• Item Type: MPRA Paper
• Language: English
• Keywords: forecasting; random walk; out-of-sample; prediction; mean square prediction error
• Subjects: C - Mathematical and Quantitative Methods > C0 - General > C01 - Econometrics; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C12 - Hypothesis Testing: General; G - Financial Economics > G1 - General Financial Markets > G17 - Financial Forecasting and Simulation

References

1. Andrews, D. W. K. (1991). Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation. Econometrica, 59(3), 817-858.

2. Busetti, F., and Marcucci, J. (2013). Comparing forecast accuracy: a Monte Carlo investigation. International Journal of Forecasting, 29(1), 13-27.

3. Campbell, J. Y., and Shiller, R. J. (1987). Cointegration and Tests of Present Value Models. Journal of Political Economy, 95(5), 1062–1088.

4. Chen, Y.-C., Rogoff, K. S., and Rossi, B. (2010). Can Exchange Rates Forecast Commodity Prices ? Quarterly Journal of Economics, 125(August), 1145–1194.

5. Chen, Y.-C., Rogoff, K. S., and Rossi, B. (2011). Predicting Agri-Commodity Prices: An Asset Pricing Approach, World Uncertainty and the Volatility of Commodity Markets, ed. B. Munier, IOS.

6. Chong, Y. Y., and Hendry, D. F. (1986). Econometric evaluation of linear macro-economic models. The Review of Economic Studies, 53(4), 671-690.

7. Clark, T. E., and McCracken, M. W. (2001). Tests of equal forecast accuracy and encompassing for nested models. Journal of Econometrics, 105, 85–110.

8. Clark, T. E., and McCracken, M. W. (2005). The power of tests of predictive ability in the presence of structural breaks. Journal of Econometrics, 124(1), 1–31.

9. Clark, T. E., and West, K. D. (2006). Using out-of-sample mean squared prediction errors to test the martingale difference hypothesis. Journal of Econometrics, 135(1–2), 155–186.

10. Clark, T. E., and West, K. D. (2007). Approximately normal tests for equal predictive accuracy in nested models. Journal of Econometrics, 138(1), 291–311.

11. Clark, T., and McCracken, M. (2013a). Advances in forecast evaluation. In Handbook of Economic Forecasting, vol. 2B., Elsevier, Amsterdam, 1107-1201.

12. Clark, T., and McCracken, M. (2013b). Evaluating the accuracy of forecasts from vector autoregressions. In: Fomby. T., Kilian, L., Murphy, A. (Eds.), Vector Autoregressive Modeling – New Developments and Applications: Essays in Honor of Christopher A. Sims, Emerald Group Publishing, Bingley.

13. Clements, M. P., and Hendry, D. F. (1993). On the limitations of comparing mean square forecast errors. Journal of Forecasting, 12(8), 617–637.

14. Diebold, F. X. (2015). Comparing Predictive Accuracy, Twenty Years Later: A Personal Perspective on the Use and Abuse of Diebold–Mariano Tests. Journal of Business and Economic Statistics, 33(1), 1-1.

15. Diebold, F. X., and Mariano, Roberto, S. (1995). Comparing Predictive Accuracy. Journal of Business and Economic Statics, 13(3), 253–263.

16. Engel, C., and West, K. D. (2005). Exchange Rates and Fundamentals. Journal of Political Economy, 113(3), 485–517.

17. Giacomini, R., and Rossi, B. (2013). Forecasting in macroeconomics. In Handbook of research methods and applications in empirical macroeconomics, Cheltenham, UK: Edward Elgar Publishing.,Chapter 17, 381-408.

18. Harvey, D. S., Leybourne, S. J., and Newbold, P. (1998). Tests for forecast encompassing. Journal of Business and Economic Statistics, 16(2), 254–259.

19. Mankiw, N. G., and Shapiro, M. D. (1986). Do we reject too often?: Small sample properties of tests of rational expectations models. Economics Letters, 20(2), 139-145.

20. McCracken, M. W. (2007). Asymptotics for out of sample tests of Granger causality. Journal of Econometrics, 140(2), 719–752.

21. Nelson, C. R., and Kim, M. J. (1993). Predictable Stock Returns: The Role of Small Sample Bias. The Journal of Finance, 48(2), 641–661.

22. Newey, W. K., and West, K. D. (1994). Automatic Lag Selection in Covariance Matrix Estimation. The Review of Economic Studies, 61(4), 631–653.

23. Newey, W. K., and West, K. D. (1987). Hypothesis testing with efficient method of moments estimation. International Economic Review, 28(3) ,777-787.

24. Pincheira, P., and Hardy, N. (2019). Forecasting Aluminum Prices with Commodity Currencies Forecasting Aluminum Prices with Commodity Currencies. Available at SSRN 3511564.

25. Pincheira, P., and Hardy, N. (2018). The predictive relationship between exchange rate expectations and base metal prices. Available at SSRN 3263709.

26. Pincheira, P., and Hardy, N. (2019). Forecasting base metal prices with the Chilean exchange rate. Resources Policy, 62(February), 256–281.

27. Pincheira, P., and West, K. D. (2016) A comparison of some out-of-sample tests of predictability in iterated multi-step-ahead forecasts. Research in Economics 70.2: 304-319.

28. Stambaugh, R. F. (1999). Predictive regressions. Journal of Financial Economics, 54(3), 375-421.

29. West, K. D. (1996). Asymptotic Inference about Predictive Ability. Econometrica, 64(5), 1067.

30. West, K. D. (2006). Chapter 3 Forecast Evaluation. Handbook of Economic Forecasting, 1(05), 99–134.