Estimating Gaussian Mixture Autoregressive model with Sequential Monte Carlo algorithm: A parallel GPU implementation

Yin, Ming (2015): Estimating Gaussian Mixture Autoregressive model with Sequential Monte Carlo algorithm: A parallel GPU implementation.

Abstract

In this paper, we propose using Bayesian sequential Monte Carlo (SMC) algorithm to estimate the univariate Gaussian mixture autoregressive (GMAR) model. The prominent benefit of the Bayesian approach is that the stationarity restriction required by the GAMR model can be straightforwardly imposed via prior distribution. In addition, compared to MCMC (Markov Chain Monte Carlo) and other simulation based algorithms, the SMC is robust to multimodal posteriors, and capable of providing fast on-line estimation when new data is available. Furthermore, it has a linear computational complexity and is ready for parallelism. To demostrate the SMC, an empirical application with US GDP growth data is considered. After estimation, we conduct the Bayesian model selection to evaluate the empirical evidence for different GMAR models. To facilitate the realization of this compute-intensive estimation, we parallelize the SMC algorithm on a nVidia CUDA compatible Graphical Process Unit (GPU) card.

• Item Type: MPRA Paper
• Language: English
• Keywords: Nonlinear Time Series, Gaussian mixture autoregressive, Sequential Monte Carlo, Particle Filter, Bayesian Inference, GPGPU, Parallel Computing
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C11 - Bayesian Analysis: General; C - Mathematical and Quantitative Methods > C3 - Multiple or Simultaneous Equation Models ; Multiple Variables > C32 - Time-Series Models ; Dynamic Quantile Regressions ; Dynamic Treatment Effect Models ; Diffusion Processes ; State Space Models; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C52 - Model Evaluation, Validation, and Selection; C - Mathematical and Quantitative Methods > C8 - Data Collection and Data Estimation Methodology ; Computer Programs > C88 - Other Computer Software

References

Boldin, M. (1992). Using Switching Models to Study Business Cycle Asymmetries: Overview of Methodology and Applications. Federal Reserve Bank of New York Research Paper, 1992-11.

Boldin, Michael. (1996). A Check on the Robustness of Hamilton's Markov Switching Model Approach to the Economic Analysis of the Business Cycle, Studies in Nonlinear Dynamics & Econometrics. 1(1), 1-14.

Camacho, M., & Perez, Q. G. (2007). Jump-and-Rest Effect of U.S. Business Cycles. Studies in Nonlinear Dynamics & Econometrics. De Gruyter. 11(4), 1-39.

Carvalho, C. M. Johannes, M. Lopes, H. F., & Polson, N. (2010). Particle learning and smoothing. Statistical Science 25, 88-106. 710-730.

Celeux, G. Hurn, M., & Robert, C.P. (2000). Computational and Inferential Difficulties with Mixture Posterior Distributions. Journal of the American Statistical Association, 95(451), 957-970.

Chopin, N. (2004). Central limit theorem for sequential Monte Carlo and its application to Bayesian inference. The Annals of Statistics, 32(6), 2385–2411.

Chopin, N. Lelievre,T., & Stoltz, G. (2012). Free energy methods for Bayesian statistics: Efficient exploration of univariate Gaussian mixture posteriors. Statistics and Computing, 22(4), 897-916.

Cogley, T., & Nason, J. M. (1995). Output Dynamics in Real-Business-Cycle Models. American Economic Review, American Economic . 85(3), 492-511.

Del Moral, P., A. Doucet, & A. Jasra (2008). On adaptive resampling procedures for sequential Monte Carlo methods. Unpublished manuscript, Insitut National de Recherche en Informatique et en Automatique, Bordeaux, France.

Doucet, A., N. de Freitas., & N. Gordon (2001). Sequential Monte Carlo Methods in Practice. New York, NY: Springer-Verlag.

Dueker, M. J. Sola, M., & Spagnolo, F. (2007). Contemporaneous threshold autoregressive models: estimation, testing and forecasting. Journal of Econometrics, 141, 517-547.

Durham,G. & Geweke, J. (2014). Adaptive sequential posterior simulators for massively parallel computing environments. Bayesian model comparison, 1-44.

Fruhwirth-Schnatter, S. (2001). Markov chain Monte Carlo estimation of classical and dynamic switching and mixture models. Journal of the American Statistical Association, 96, 194-209.

Geweke, J. (1989). Bayesian inference in econometric models using Monte Carlo integration. Econometrica, 57, 1317-1339.

Geweke, J. (2007). Interpretation and inference in mixture models: Simple MCMC works. Computational Statistics and Data Analysis, 51(7), 3529 - 3550.

Gordon, N. J. Salmond, D. J., & Smith, A. F. M. (1993). Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEEE Proceedings F on Radar and Signal Processing, 140(2), 107–113.

Gourieroux, C., & Robert, C. Y. (2006). Stochastic unit root models. Econometric Theory, 22, 1052-1090.

Hamilton, J., (1989). A New Approach to Economic Analysis of Nonstationary Time Series. Econometrica, 57, 357–384.

Herbst, E., & F. Schorfheide. (2014). Sequential Monte Carlo sampling for DSGE models. Journal of Applied Econometrics, 29, 1073–1098.

Kass, R., & Raftery, A. (1995). Bayes Factors. Journal of the American Statistical Association, 90(430), 773-795.

Kalliovirta, L., & Meitz, M and Saikkonen, P. (2015). A Gaussian mixture autoregressive model for univariate time series. Journal of Time Series Analysis, 36(2), 247-266.

Kim, C. J., & C. J. Murray (2002). Permanent and transitory components of recessions. Empirical Economics 27,(2), 163-183.

Kim, C. J., Morley, J., & J. Piger. (2005). Nonlinearity and the Permanent Effects of Recessions, Journal of Applied Econometrics 20, 291-309.