Variational Bayes inference in high-dimensional time-varying parameter models

Koop, Gary and Korobilis, Dimitris (2018): Variational Bayes inference in high-dimensional time-varying parameter models.

Abstract

This paper proposes a mean field variational Bayes algorithm for efficient posterior and predictive inference in time-varying parameter models. Our approach involves: i) computationally trivial Kalman filter updates of regression coefficients, ii) a dynamic variable selection prior that removes irrelevant variables in each time period, and iii) a fast approximate state-space estimator of the regression volatility parameter. In an exercise involving simulated data we evaluate the new algorithm numerically and establish its computational advantages. Using macroeconomic data for the US we find that regression models that combine time-varying parameters with the information in many predictors have the potential to improve forecasts over a number of alternatives.

• Item Type: MPRA Paper
• Language: English
• Keywords: dynamic linear model; approximate posterior inference; dynamic variable selection; forecasting
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C11 - Bayesian Analysis: General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C13 - Estimation: General; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C52 - Model Evaluation, Validation, and Selection; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C53 - Forecasting and Prediction Methods ; Simulation Methods; C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling > C61 - Optimization Techniques ; Programming Models ; Dynamic Analysis

References

Angelino, E., Johnson, M. J. and R. P. Adams (2016). Patterns of scalable Bayesian inference. Foundations and Trends$^{\text{\textregistered$ in Machine Learning 9(2-3), 119-247.

Bauwens, L., Koop, G., Korobilis, D. and J. V. K. Rombouts (2015). The contribution of structural break models to forecasting macroeconomic series. Journal of Applied Econometrics 30(4), 596-620.

Beal, M. J. (2003). Variational algorithms for approximate Bayesian inference. PhD Thesis, Gatsby Computational Neuroscience Unit, University College London.

Beal, M. J. and Z. Ghahramani (2003). The Variational Bayesian EM Algorithm for Incomplete Data: with Application to Scoring Graphical Model Structures. In Bernardo, J.M., Dawid, A.P., Berger, J.O., West, M., Heckerman, D and, Bayarri, M.J., (Eds.) Bayesian Statistics 7 453-464, Oxford University Press.

Belmonte, M., Koop, G. and D. Korobilis (2014). Hierarchical shrinkage in time-varying coefficients models. Journal of Forecasting 33, 80-94.

Blei, D., Kucukelbir, A. and J. McAuliffe (2017). Variational inference: A review for statisticians. Journal of the American Statistical Association 112, 859-877.

Chan, J., Koop, G., Leon-Gonzalez, R. and R. Strachan (2012). Time varying dimension models. Journal of Business and Economic Statistics 30(3), 358-367.

Clark, T. E. and F. Ravazzolo (2015). Macroeconomic forecasting performance under alternative specifications of time‐varying volatility. Journal of Applied Econometrics 30(4), 551-575.

Cooley, T. F. and E. C. Prescott (1976). Estimation in the presence of stochastic parameter variation. Econometrica 44(1), 167-184.

Dangl, T. and M. Halling (2012). Predictive regressions with time-varying coefficients. Journal of Financial Economics 106, 157-181.

Dempster, A. P., Laird, N. M. and D. B. Rubin (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B (Methodological) 39, 1-38.

Fruehwirth-Schnatter, S. and H. Wagner (2010). Stochastic model specification search for Gaussian and partial non-Gaussian state space models. Journal of Econometrics 154(1), 85-100.

George, E. I. and R. E. McCulloch (1993). Variable selection via Gibbs sampling. Journal of the American Statistical Association 88, 881-889.

George, E. I. and R. E. McCulloch (1997). Approaches for Bayesian variable selection. Statistica Sinica 7, 339-373.

Giordani, P. and R. Kohn (2008). Efficient Bayesian inference for multiple change-point and mixture innovation models. Journal of Business and Economic Statistics, 26(1) 66-77.

Groen, J. J. J., Paap, R. and F. Ravazzollo (2013). Real time inflation forecasting in a changing world. Journal of Business and Economic Statistics, 31(1) 29-44.

Hajargasht, G. and T. Wozniak (2018). Variational Bayes inference for large vector autoregressions. Manuscript.

Kalli, M. and J. E. Griffin (2014). Time-varying sparsity in dynamic regression models. Journal of Econometrics 178, 779-793.

Koop, G. and D. Korobilis (2012). Forecasting inflation using dynamic model averaging. International Economic Review 53, 867-886.

Koop, G. and S. Potter (2007). Estimation and forecasting in models with multiple breaks. Review of Economic Studies 74(3), 763-789.

Kowal, D. R., Matteson, D. S. and D. Ruppert (2017). Dynamic shrinkage processes. arXiv:1707.00763.

Kulhavy, R. and F. Kraus (1996). On duality of regularized exponential and linear forgetting. Automatica 32,1403-1416.

Kuo, L., and B. Mallick (1997). Variable selection for regression models. Shankya: The Indian Journal of Statistics 60 (Series B), 65-81.

Nakajima, J. and M. West (2013). Bayesian analysis of latent threshold dynamic models. Journal of Business and Economic Statistics 31, 151–164.

Ormerod, J. and M. Wand (2010). Explaining variational approximations. American Statistician 64, 140-153.

Pettenuzzo, D. and A. Timmermann (2017). Forecasting macroeconomic variables under model instability. Journal of Business and Economic Statistics 35(2), 183-201.

Raftery, A., Karny, M., and P. Ettler (2010), Online prediction under model uncertainty via dynamic model averaging: Application to a cold rolling mill. Technometrics 52, 52-66.

Rockova, V. and K. McAlinn (2018). Dynamic variable selection with spike-and-slab process priors. Technical report, Booth School of Business, University of Chicago.

Sarris, A. H. (1973). Kalman filter models: A Bayesian approach to estimation of time-varying regression coefficients. In: Berg, V. G. (Ed.) Annals of Economic and Social Measurement 2(4), 501-523. NBER: Cambridge, MA.

Shumway, R. H. and D. S. Stoffer (1982). An approach to time series smoothing and forecasting using the EM algorithm. Journal of Time Series Analysis 3(4), 253-264.

Stock, J. H. and M. W. Watson (2002). Macroeconomic forecasting using diffusion indexes. Journal of Business and Economic Statistics 20(2), 147-162.

Stock, J. H. and M. W. Watson (2007). Why has U.S. inflation become harder to forecast? Journal of Money, Credit and Banking 39(1), 3-33.

Uribe, P. V. and H. F. Lopes (2017). Dynamic sparsity on dynamic regression models. Manuscript, available at http://hedibert.org/wp-content/uploads/2018/06/uribe-lopes-Sep2017.pdf.

Wand, M. P. (2017). Fast approximate inference for arbitrarily large semiparametric regression models via message passing. Journal of the American Statistical Association 112, 137-168.

Wang, H., Yu, H., Hoy, M., Dauwels, J. , and H. Wang (2016). Variational Bayesian dynamic compressive sensing, 2016 IEEE International Symposium on Information Theory.

Wang, Y. and D. M. Blei (forthcoming). Frequentist consistency of variational Bayes. Journal of the American Statistical Association, https://doi.org/10.1080/ 01621459.2018.1473776.