High-dimensional macroeconomic forecasting using message passing algorithms

Korobilis, Dimitris (2019): High-dimensional macroeconomic forecasting using message passing algorithms.

Abstract

This paper proposes two distinct contributions to econometric analysis of large information sets and structural instabilities. First, it treats a regression model with time-varying coefficients, stochastic volatility and exogenous predictors, as an equivalent high-dimensional static regression problem with thousands of covariates. Inference in this specification proceeds using Bayesian hierarchical priors that shrink the high-dimensional vector of coefficients either towards zero or time-invariance. Second, it introduces the frameworks of factor graphs and message passing as a means of designing efficient Bayesian estimation algorithms. In particular, a Generalized Approximate Message Passing (GAMP) algorithm is derived that has low algorithmic complexity and is trivially parallelizable. The result is a comprehensive methodology that can be used to estimate time-varying parameter regressions with arbitrarily large number of exogenous predictors. In a forecasting exercise for U.S. price inflation this methodology is shown to work very well.

• Item Type: MPRA Paper
• Language: English
• Keywords: high-dimensional inference; factor graph; Belief Propagation; Bayesian shrinkage; time-varying parameter model
• Subjects: C - Mathematical and Quantitative Methods > C0 - General > C01 - Econometrics; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C11 - Bayesian Analysis: General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C13 - Estimation: General; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C52 - Model Evaluation, Validation, and Selection; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C53 - Forecasting and Prediction Methods ; Simulation Methods; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C55 - Large Data Sets: Modeling and Analysis; C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling > C61 - Optimization Techniques ; Programming Models ; Dynamic Analysis; E - Macroeconomics and Monetary Economics > E3 - Prices, Business Fluctuations, and Cycles > E31 - Price Level ; Inflation ; Deflation; E - Macroeconomics and Monetary Economics > E3 - Prices, Business Fluctuations, and Cycles > E37 - Forecasting and Simulation: Models and Applications

References

Al-Shoukairi, M., Schniter, P. and B. D. Rao (2018). A GAMP-based low complexity sparse Bayesian learning algorithm. IEEE Transactions on Signal Processing, 66(2), 294-308.

Amir-Ahmadi, P., Matthes, C. and M.-C. Wang (forthcoming). Choosing prior hyperparameters: With applications to time-varying parameter models. Journal of Business and Economic Statistics.

Angelino, E., Johnson, M. J. and R. P. Adams (2016). Patterns of scalable Bayesian inference. Foundations and Trends in Machine Learning, 9(2-3), 119-247.

Barber, D. (2012). Bayesian reasoning and machine learning. Cambridge University Press: New York.

Bauwens, L., Koop, G., Korobilis, D. and J. V. K. Rombouts (2015). The contribution of structural break models to forecasting macroeconomic series. Journal of Applied Econometrics, 30(4), 596-620.

Belmonte, M., Koop, G. and D. Korobilis (2014). Hierarchical shrinkage in time-varying coefficients models. Journal of Forecasting, 33, 80-94.

Bishop, C. M. (2006). Pattern recognition and machine learning. Springer: New York.

Chan, J., Koop, G., Leon-Gonzalez, R. and R. Strachan (2012). Time varying dimension models. Journal of Business and Economic Statistics 30(3), 358-367.

Cooley, T. F. and E. C. Prescott (1976). Estimation in the presence of stochastic parameter variation. Econometrica 44(1), 167-184.

Donoho, D. L., Maleki, A. and A. Montanari (2009). Message passing algorithms for compressed sensing. Proceedings of National Academy of Sciences, 106(45), 18914-18919.

Donoho, D. L., Maleki, A. and A. Montanari (2011). How to design message passing algorithms for compressed sensing. Unpublished manuscript, available at http://www.ece.rice.edu/~mam15/bpist.pdf.

Fruehwirth-Schnatter, S. and H. Wagner (2010). Stochastic model specification search for Gaussian and partial non-Gaussian state space models. Journal of Econometrics 154(1), 85-100.

George, E. I. and R. E. McCulloch (1993). Variable selection via Gibbs sampling. Journal of the American Statistical Association, 88(423), 881-889.

Giordani, P. and R. Kohn (2008). Efficient Bayesian inference for multiple change-point and mixture innovation models. Journal of Business and Economic Statistics, 26(1) 66-77.

Groen, J. J. J., Paap, R. and F. Ravazzollo (2013). Real time inflation forecasting in a changing world. Journal of Business and Economic Statistics, 31(1) 29-44.

Herbst, E. P. and F. Schorfheide (2015). Bayesian estimation of DSGE models. Princeton University Press: New Jersey.

Jackson, M. O. (2008). Social and economic networks. Princeton University Press: New Jersey.

Kalli, M. and J. E. Griffin (2014). Time-varying sparsity in dynamic regression models. Journal of Econometrics 178, 779-793.

Koop, G. and D. Korobilis (2010). Bayesian multivariate time series methods for empirical macroeconomics. Foundations and Trends in Econometrics 3, pp. 267-358.

Korobilis, D. (2013). Hierarchical shrinkage priors for dynamic regressions with many predictors. International Journal of Forecasting, 29, 43-59.

Kowal, D. R., Matteson, D. S. and D. Ruppert (2017). Dynamic shrinkage processes. arXiv:1707.00763.

Kschischang, F. R., Frey, B. J. and H. A. Loeliger (2001). Factor graphs and the sum-product algorithm. IEEE Transactions on Information Theory, 47(2), 498-519.

Mooij, J. and H. Kappen (2007). Sufficient conditions for convergence of the sum–product algorithm. IEEE Transactions on Information Theory, 53(12), 4422-4437.

Nakajima, J. and M. West (2013). Bayesian analysis of latent threshold dynamic models. Journal of Business and Economic Statistics, 31(2), 151-164.

Pearl, J. (1982). Reverend Bayes on inference engines: A distributed hierarchical approach. AAAI-82: Pittsburgh, PA. Second National Conference on Artificial Intelligence. Menlo Park, California: AAAI Press, 133-136.

Pettenuzzo, D. and A. Timmermann (2017). Forecasting macroeconomic variables under model instability. Journal of Business and Economic Statistics, 35(2), 183-201.

Rangan, S. (2011). Generalized approximate message passing for estimation with random linear mixing. IEEE International Symposium on Information Theory, 2174-2178.

Rangan, S., Schniter, P., Riegler, E., Fletcher, A. K. and V. Cevher (2016). Fixed points of generalized approximate message passing with arbitrary matrices. arXiv:1301.6295v4.

Rockova, V. and K. McAlinn (2018). Dynamic variable selection with spike-and-slab process priors. Technical report, Booth School of Business, University of Chicago.

Stock, J. H. and M. W. Watson (1999). Forecasting inflation. Journal of Monetary Economics, 44(2), 293-335.

Stock, J. H. and M. W. Watson (2007). Why has U.S. inflation become harder to forecast? Journal of Money, Credit and Banking 39, 3–33.

Tipping, M. E. (2001). Sparse Bayesian learning and the relevance vector machine. Journal of Machine Learning Research, 1, 211-244.

Vehtari, A., Gelman, A., Sivula, T., Jylanki, P., Tran, D., Sahai, S., Blomstedt, P., Cunningham, J. P., Schiminovich, D., and C. Robert (2018). Expectation propagation as a way of life: A framework for Bayesian inference on partitioned data. arXiv:1412.4869v3.

Wand, M. P. (2017). Fast approximate inference for arbitrarily large semiparametric regression models via message passing. Journal of the American Statistical Association} 112, 137-168.

Zou, X., Li, F., Fang, J. and H. Li(2016). Computationally efficient sparse Bayesian learning via generalized approximate message passing. IEEE International Conference on Ubiquitous Wireless Broadband (ICUWB), 1-4.