Bayesian dynamic variable selection in high dimensions

Korobilis, Dimitris and Koop, Gary (2020): Bayesian dynamic variable selection in high dimensions.

Abstract

This paper proposes a variational Bayes algorithm for computationally efficient posterior and predictive inference in time-varying parameter (TVP) models. Within this context we specify a new dynamic variable/model selection strategy for TVP dynamic regression models in the presence of a large number of predictors. This strategy allows for assessing in individual time periods which predictors are relevant (or not) for forecasting the dependent variable. The new algorithm is evaluated numerically using synthetic data and its computational advantages are established. Using macroeconomic data for the US we find that regression models that combine time-varying parameters with the information in many predictors have the potential to improve forecasts of price inflation over a number of alternative forecasting models.

• Item Type: MPRA Paper
• Language: English
• Keywords: dynamic linear model; approximate posterior inference; dynamic variable selection; forecasting
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C11 - Bayesian Analysis: General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C13 - Estimation: General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C15 - Statistical Simulation Methods: General; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C22 - Time-Series Models ; Dynamic Quantile Regressions ; Dynamic Treatment Effect Models ; Diffusion Processes; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C52 - Model Evaluation, Validation, and Selection; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C53 - Forecasting and Prediction Methods ; Simulation Methods; C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling > C61 - Optimization Techniques ; Programming Models ; Dynamic Analysis; C - Mathematical and Quantitative Methods > C8 - Data Collection and Data Estimation Methodology ; Computer Programs > C87 - Econometric Software

References

Bauwens, L., G. Koop, D. Korobilis, and J. V. Rombouts (2015): The Contribution of Structural Break Models to Forecasting Macroeconomic Series, Journal of Applied Econometrics, 30, 596-620.

Beal, M. J. and Z. Ghahramani (2003): The Variational Bayesian EM Algorithm for Incomplete Data With Application to Scoring Graphical Model Structures, in Bayesian Statistics, ed. by J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith, and M. West, Oxford: Oxford University Press, vol. 7, 453-464.

Belmonte, M. A., G. Koop, and D. Korobilis (2014): Hierarchical Shrinkage in Time-Varying Parameter Models, Journal of Forecasting, 33, 80-94.

Bitto, A. and S. Fr�uhwirth-Schnatter (2019): Achieving Shrinkage in a Time- Varying Parameter Model Framework, Journal of Econometrics, 210, 75-97, annals Issue in Honor of John Geweke Complexity and Big Data in Economics and Finance: Recent Developments from a Bayesian Perspective".

Blei, D. M., A. Kucukelbir, and J. D. McAuliffe (2017): Variational Inference: A Review for Statisticians, Journal of the American Statistical Association, 112, 859-877.

Breiman, L. (1996): Bagging Predictors, Machine Learning, 24, 123-140.

Byrne, J. P., D. Korobilis, and P. Ribeiro (2018): On the Sources of Uncertainty in Exchange Rate Predictability, International Economic Review, 59, 329-357.

Callot, L. and J. T. Kristensen (2014): Vector Autoregressions with parsimoniously Time Varying Parameters and an Application to Monetary Policy, Tinbergen Institute Discussion Papers 14-145/III, Tinbergen Institute.

Chan, J. and I. Jeliazkov (2009): E�cient simulation and integrated likelihood estimation in state space models, International Journal of Mathematical Modelling and Numerical Optimisation, 1, 101-120.

Clark, T. E. and F. Ravazzolo (2015): Macroeconomic Forecasting Performance under Alternative Speci�cations of Time-Varying Volatility, Journal of Applied Econometrics, 30, 551-575.

Cogley, T. and T. J. Sargent (2005): Drifts and volatilities: monetary policies and outcomes in the post WWII US, Review of Economic Dynamics, 8, 262-302, monetary Policy and Learning.

Cooley, T. F. and E. C. Prescott (1976): Estimation in the Presence of Stochastic Parameter Variation, Econometrica, 44, 167-184.

Dangl, T. and M. Halling (2012): Predictive Regressions with Time-Varying Coe�cients, Journal of Financial Economics, 106, 157-181.

De Mol, C., D. Giannone, and L. Reichlin (2008): Forecasting using a large number of predictors: Is Bayesian shrinkage a valid alternative to principal components?" Journal of Econometrics, 146, 318-328, honoring the research contributions of Charles R. Nelson.

Dempster, A. P., N. M. Laird, and D. B. Rubin (1977): Maximum Likelihood from Incomplete Data via the EM Algorithm, Journal of the Royal Statistical Society. Series B (Methodological), 39, 1-38.

Farmer, L., L. Schmidt, and A. G. Timmermann (2018): Pockets of Predictability, CEPR Discussion Papers 12885, C.E.P.R. Discussion Papers.

George, E. I. and R. E. McCulloch (1993): Variable Selection via Gibbs Sampling, Journal of the American Statistical Association, 88, 881-889.

Giannone, D., M. Lenza, and G. E. Primiceri (2017): Economic Predictions with Big Data: The Illusion Of Sparsity, CEPR Discussion Papers 12256, C.E.P.R. Discussion Papers.

Granger, C. (2008): Non-Linear Models: Where Do We Go Next - Time Varying Parameter Models?" Studies in Nonlinear Dynamics & Econometrics, 12, 1-9.

Jurado, K., S. C. Ludvigson, and S. Ng (2015): Measuring Uncertainty, American Economic Review, 105, 1177-1216.

Kalli, M. and J. E. Griffin (2014): Time-Varying Sparsity in Dynamic Regression Models, Journal of Econometrics, 178, 779-793.

Koop, G. and D. Korobilis (2012): Forecasting In ation Using Dynamic Model Averaging, International Economic Review, 53, 867-886.

Korobilis, D. (2019): High-Dimensional Macroeconomic Forecasting Using Message Passing Algorithms, Journal of Business & Economic Statistics, 0, 1-12.

Kowal, D. R., D. S. Matteson, and D. Ruppert (2019): Dynamic Shrinkage Processes, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 81, 781-804.

MATLAB (2020): MATLAB Statistics and Machine Learning Toolbox, The MathWorks, Natick, MA, USA.

McCracken, M. and S. Ng (2020): FRED-QD: A Quarterly Database for Macroeconomic Research, Working Paper 26872, National Bureau of Economic Research.

Naesseth, C. A., S. W. Linderman, R. Ranganath, and D. M. Blei (2017): Variational Sequential Monte Carlo, .

Nakajima, J. and M. West (2013): Bayesian Analysis of Latent Threshold Dynamic Models, Journal of Business & Economic Statistics, 31, 151-164.

Narisetty, N. N. and X. He (2014): Bayesian variable selection with shrinking and diffusing priors, The Annals of Statistics, 42, 789-817.

Ormerod, J. T. and M. P. Wand (2010): Explaining Variational Approximations, The American Statistician, 64, 140-153.

Rossi, B. (2013): Chapter 21 - Advances in Forecasting under Instability, in Handbook of Economic Forecasting, ed. by G. Elliott and A. Timmermann, Elsevier, vol. 2, 1203 - 1324.

Rockova, V. and K. McAlinn (2017): Dynamic Variable Selection with Spike-and-Slab Process Priors, Tech. Rep. arXiv:1708.00085v2, ArXiV.

Smidl, V. and A. Quinn (2006): The Variational Bayes Method in Signal Processing, Signals and Communication Technology, Springer.

Stock, J. H. and M. W. Watson (2007): Why Has U.S. Inflation Become Harder to Forecast?" Journal of Money, Credit and Banking, 39, 3-33.

Stock, J. H. and M. W. Watson (2016): Chapter 8 - Dynamic Factor Models, Factor-Augmented Vector Autoregressions, and Structural Vector Autoregressions in Macroeconomics, in Handbook of Macroeconomics, ed. by J. B. Taylor and H. Uhlig, Elsevier, vol. 2, 415 - 525.

Tran, M.-N., D. J. Nott, and R. Kohn (2017): Variational Bayes With Intractable Likelihood, Journal of Computational and Graphical Statistics, 26, 873-882.

Uhlig, H. (1994): On Singular Wishart and Singular Multivariate Beta Distributions, Ann. Statist., 22, 395-405.

Uribe, P. and H. Lopes (2017): Dynamic Sparsity on Dynamic Regression Models, Tech. rep., Available at http://hedibert.org/wp-content/uploads/2018/06/uribe-lopes- Sep2017.pdf.

Wang, H., H. Yu, M. Hoy, J. Dauwels, and H. Wang (2016): Variational Bayesian Dynamic Compressive Sensing, in 2016 IEEE International Symposium on Information Theory (ISIT), 1421-1425.

Wang, Y. and D. M. Blei (2019): Frequentist Consistency of Variational Bayes, Journal of the American Statistical Association, 114, 1147-1161.

Welch, I. and A. Goyal (2007): A Comprehensive Look at The Empirical Performance of Equity Premium Prediction, The Review of Financial Studies, 21, 1455-1508.

West, M. and J. Harrison (1997): Bayesian Forecasting and Dynamic Models (2nd ed.), Berlin, Heidelberg: Springer-Verlag.

Yousuf, K. and S. Ng (2019): Boosting High Dimensional Predictive Regressions with Time Varying Parameters, Tech. Rep. arXiv:1910.03109, ArXiV.

Zou, H. and T. Hastie (2005): Regularization and Variable Selection via the Elastic Net, Journal of the Royal Statistical Society. Series B (Statistical Methodology), 67, 301-320.