Hierarchical shrinkage priors for dynamic regressions with many predictors

Korobilis, Dimitris (2011): Hierarchical shrinkage priors for dynamic regressions with many predictors.

Abstract

This paper builds on a simple unified representation of shrinkage Bayes estimators based on hierarchical Normal-Gamma priors. Various popular penalized least squares estimators for shrinkage and selection in regression models can be recovered using this single hierarchical Bayes formulation. Using 129 U.S. macroeconomic quarterly variables for the period 1959 -- 2010 I exhaustively evaluate the forecasting properties of Bayesian shrinkage in regressions with many predictors. Results show that for particular data series hierarchical shrinkage dominates factor model forecasts, and hence it becomes a valuable addition to existing methods for handling large dimensional data.

• Item Type: MPRA Paper
• Language: English
• Keywords: Forecasting; shrinkage; factor model; variable selection; Bayesian LASSO
• Subjects: C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C53 - Forecasting and Prediction Methods ; Simulation Methods; C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling > C63 - Computational Techniques ; Simulation Modeling; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C52 - Model Evaluation, Validation, and Selection; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C22 - Time-Series Models ; Dynamic Quantile Regressions ; Dynamic Treatment Effect Models ; Diffusion Processes; E - Macroeconomics and Monetary Economics > E3 - Prices, Business Fluctuations, and Cycles > E37 - Forecasting and Simulation: Models and Applications; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C11 - Bayesian Analysis: General

References

Armagan, A. and Zaretzki, R. L. (2010). Model Selection via Adaptive Shrinkage with t Priors. Computational Statistics 25, 441-461.

Bai, J. and Ng, S. (2007). Boosting Diffusion Indexes. Unpublished manuscript, Columbia University.

De Mol, C., Giannone, D. and Reichlin, L. (2008). Forecasting Using a Large Number of Predictors: Is Bayesian Shrinkage a Valid Alternative to Principal Components? Journal of Econometrics 146, 318-328.

Efron, B., Hastie, T., Johnstone, I. and Tibshirani, R. (2004). Least Angle Regression. The Annals of Statistics 32, 407-451.

Efron, B. and Morris, C. (1975). Data Analysis Using Stein's Estimator and its Generalizations. Journal of the American Statistical Association 70, 311-319.

Fernandez, C., Ley, E. and Steel, M. F. J. (2001). Benchmark Priors for Bayesian Model Averaging. Journal of Econometrics 100, 381-427.

Gelman, A. (2006). Prior Distributions for Variance Parameters in Hierarchical Models. Bayesian Analysis 1, 515-533.

George, E and McCullogh, R. (1993). Variable Selection via Gibbs Sampling. Journal of the American Statistical Association 88, 881-889.

Geweke, J. (1993). Bayesian Treatment of the Independent Student-t Linear Model. Journal of Applied Econometrics 8, 19-40.

Geweke, J. and Amisano, G. (2010). Comparing and evaluating Bayesian predictive distributions of asset returns. International Journal of Forecasting 26, 216-230.

Hobert, J. P. and Casella, G. (1996). The Effect of Improper Priors on Gibbs Sampling in Hierarchical Mixed Models. Journal of the American Statistical Association 91, 1461-1473.

Hobert, J. P. and Geyer, C. J. (1998). Geometric Ergodicity of Gibbs and Block Gibbs Samplers for a Hierarchical Random Effects Model. Journal of Multivariate Analysis 67, 414-430.

Inoue, A. and Kilian, L. (2008). How Useful Is Bagging in Forecasting Economic Time Series? A Case Study of U.S. Consumer Price Inflation. Journal of the American Statistical Association 103, 511-522.

Judge, G. G. and Bock, M.E. (1978). Statistical Implications of Pre-Test and Stein Rule Estimators in Econometrics. North-Holland, Amsterdam.

Koop, G. (2003). Bayesian Econometrics. Wiley, Chichester.

Koop, G. and Korobilis, D. (2009). Forecasting Inflation Using Dynamic Model Averaging, Working Paper Series 34-09, Rimini Centre for Economic Analysis.

Koop, G. and Potter, S. (2004). Forecasting in Dynamic Factor Models Using Bayesian Model Averaging. The Econometrics Journal 7, 550-565.

Kyung, M., Gill, J., Ghoshz, M. and Casella, G. (2010). Penalized Regression, Standard Errors, and Bayesian Lassos. Bayesian Analysis 5, 369-412.

Liang, F., Paulo, R., Molina, G., Clyde, M. A. and Berger, J. O. (2008). Mixtures of g-priors for Bayesian Variable Selection. Journal of the American Statistical Association 103, 410-423.

Litterman, R. (1979). Techniques of forecasting using vector autoregressions. Federal Reserve Bank of Minneapolis Working Paper 115.

Maruyama, Y. and George, E. I. (2010). gBF: A Fully Bayes Factor with a Generalized g-prior. Technical Report, University of Pennsylvania, available at http://arxiv.org/abs/0801.4410

Park, T. and Casella, G. (2008). The Bayesian Lasso. Journal of the American Statistical Association 103, 681-686.

Rosset, S. and Zhu, J. (2004), Discussion of "Least Angle Regression", by B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani. The Annals of Statistics 32, 469--475

Stock, J. and Watson, M. (2011). Generalized Shrinkage Methods for Forecasting Using Many Predictors. Unpublished Manuscipt, available at http://www.princeton.edu/~mwatson/.

Tibshirani, R. (1996). Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society, Series B 58, 267-288.

Tibshirani, R., Saunders, M., Rosset, S., Zhu, J. and Knight, K. (2005). Sparsity and Smoothness via the Fused Lasso. Journal of the Royal Statistical Society, Series B 67, 91-108.

Zellner, A. (1986). On Assessing Prior Distributions and Bayesian Regression Analysis with g-prior Distributions. in: P. Goel and A. Zellner (Eds.) Bayesian Inference and Decision Techniques (North-Holland, Amsterdam).

Zou, H. (2006). The Adaptive Lasso and Its Oracle Properties. Journal of the American Statistical Association 101, 1418-1429.

Zou, H. and Hastie, T. (2005). Regularization and Variable Selection via the Elastic Net. Journal of the Royal Statistical Society, Series B 67, 301-320.