How informative are in-sample information criteria to forecasting? the case of Chilean GDP

Medel, Carlos A. (2012): How informative are in-sample information criteria to forecasting? the case of Chilean GDP.

Abstract

There is no standard economic forecasting procedure that systematically outperforms the others at all horizons and with any dataset. A common way to proceed, in many contexts, is to choose the best model within a family based on a fitting criteria, and then forecast. I compare the out-of-sample performance of a large number of autoregressive integrated moving average (ARIMA) models with some variations, chosen by three commonly used information criteria for model building: Akaike, Schwarz, and Hannan-Quinn. I perform this exercise to identify how to achieve the smallest root mean squared forecast error with models based on information criteria. I use the Chilean GDP dataset, estimating with a rolling window sample to generate one- to four-step ahead forecasts. Also, I examine the role of seasonal adjustment and the Easter effect on out-of-sample performance. After the estimation of more than 20 million models, the results show that Akaike and Schwarz are better criteria for forecasting purposes where the traditional ARMA specification is preferred. Accounting for the Easter effect improves the forecast accuracy only with seasonally adjusted data, and second-order stationarity is best.

• Item Type: MPRA Paper
• Language: English
• Keywords: data mining; forecasting; ARIMA; seasonal adjustment; Easter-effect
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C13 - Estimation: General; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C53 - Forecasting and Prediction Methods ; Simulation Methods; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C52 - Model Evaluation, Validation, and Selection; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C22 - Time-Series Models ; Dynamic Quantile Regressions ; Dynamic Treatment Effect Models ; Diffusion Processes

References

1.Akaike, H., 1974, "A New Look at the Statistical Model Identification," IEEE Transactions on Automatic Control 19(6): 716-723.

2.Bell,W.R., 1995, "Seasonal Adjustment to Facilitate Forecasting - Arguments for Not Revising Seasonally Adjusted Data," American Statistical Association, Proceedings of the Business and Economics Statistics Section: 268-273.

3.Bell, W.R. and S.C. Hillmer, 1983, "Modelling Time Series with Calendar Variation," Journal of the American Statistical Association 78: 526-534.

4.Bell, W.R. and E. Sotiris, 2010, "Seasonal Adjustment to Facilitate Forecasting: Empirical Results," manuscript, US Census Bureau, Research Staff Papers.

5.Box, G. and G. Jenkins, 1970, Time Series Analysis: Forecasting and Control, Holden-Day, San Francisco, USA.

6.Capistrán, C., C. Constandse, and M. Ramos-Francia, 2010, "Multi-horizon Inflation Forecasts using Disaggregated Data," Economic Modelling 27(3): 666-677.

7.Clark, T.E., 2004, "Can Out-of-Sample Forecast Comparisons Help Prevent Overfitting?," Journal of Forecasting 23(2): 115-139.

8.Clements, M.P. and A.B. Galvão, 2010, "Real-time Forecasting of Inflation and Output Growth in the Presence of Data Revisions," The Warwick Economics Research Paper Series 953, Department of Economics, University of Warwick, UK.

9.Cobb, M., 2009, "Forecasting Chilean Inflation from Disaggregate Components," Working Paper 545, Central Bank of Chile.

10.Cobb, M. and C.A. Medel, 2010, "Una Estimación del Impacto del Efecto Calendario en Series Desestacionalizadas Chilenas de Actividad y Demanda," Journal Economía Chilena (The Chilean Economy) 13(3): 95-103.

11.Findley, D.F., B.C. Monsell, W.R. Bell, M.C. Otto, and B. Chen, 1998, "New Capabilities and Methods of the X12-ARIMA Seasonal Adjustment Program," Journal of Business and Economics Statistics 16(2): 127-152.

12.Ghysels, E., D. Osborn, and P.M.M. Rodrigues, 2006, Forecasting Seasonal Time Series, in Elliot, G., C.W.J. Granger, and A. Timmermann (eds.), Handbook of Economic Forecasting, Elsevier, North Holland.

13.Goodwin,P., D. Önkal, and M. Lawrence, 2011, Improving the Role of Judgement in Economic Forecasting, in Clements, M. and D.F. Hendry (eds.), The Oxford Handbook of Economic Forecasting, Oxford University Press.

14.Granger, C.W.J., 1979, Seasonality: Causation, Interpretation, and Implications, in A. Zellner (ed.), Seasonal Analysis of Economic Time Series, National Bureau of Economic Research.

15.Granger, C.W.J. and Y. Jeon, 2001, "The Roots of US Macro Time Series," Working Paper, University of California at San Diego.

16.Granger, C.W.J. and Y. Jeon, 2004, "Forecasting Performance of Information Criteria with Many Macro Series," Journal of Applied Statistics 31(10): 1227-1240.

17.Hannan, E.J. and B.G. Quinn, 1979, "The Determination of the Order of an Autoregression," Journal of the Royal Statistical Society B 41: 190-195.

18.Hansen, P.R., 2005, "A Test for Superior Predictive Ability," Journal of Business and Economic Statistics 23: 365-380.

19.Hyndman, R.J., R.A. Ahmed, G. Athanasopoulos, and H.L. Shang, 2011, "Optimal Combination Forecasts for Hierarchical Time Series," Computational Statistics and Data Analysis 55(9): 2579-2589.

20.Holan, S.H., R. Lund, and G. Davis, 2010, "The ARMA Alphabet Soup: A Tour of ARMA Model Variants," Statistics Surveys 4: 232-274.

21.Medel, C.A. and M. Urrutia, 2010, "Aggregated and Disaggregated Forecast of Chilean GDP with Automatic Time Series Procedures," (in Spanish) Working Paper 577, Central Bank of Chile.

22.Mélard, G. and J.-M. Pasteels, 2000, "Automatic ARIMA Modeling Including Interventions, using Time Series Expert Software," International Journal of Forecasting 16(4): 497-508.

23.Patton, A. and A. Timmermann, 2010, "New Tests of Forecast Optimality Across Horizons," Working Paper, University of California at San Diego.

24.Pincheira,P., 2011, "A Bunch of Models, a Bunch of Nulls and Inference About Predictive Ability," Working Paper 607, Central Bank of Chile.

25.Pincheira, P. and A. García, 2009, "Forecasting Inflation in Chile with an Accurate Benchmark," Working Paper 514, Central Bank of Chile.

26.Schwarz, G.E., 1978, "Estimating the Dimension of a Model," Annals of Statistics 6(2): 461-464.

27.Stock, J. and M. Watson, 2007, A Comparison of Linear and Nonlinear Univariate Models for Forecasting Macroeconomic Time Series, in Engle, R.E. and H. White (eds.), Cointegration, Causality, and Forecasting: A Fetstschrift in Honour of Clive W.J. Granger, Oxford University Press.

28.US Census Bureau, 2007, X12-ARIMA version 0.3 Reference Manual, http://www.census.gov/ts/x12a/v03/x12adocV03.pdf.

29.White, H., 2000, "A Reality Check for Data Snooping," Econometrica 68: 1097-1126.