A closed-form filter for binary time series

Fasano, Augusto and Rebaudo, Giovanni and Durante, Daniele and Petrone, Sonia (2021): A closed-form filter for binary time series. Published in: Statistics and Computing , Vol. 31, : pp. 1-21.

Abstract

Non-Gaussian state-space models arise in several applications, and within this framework the binary time series setting provides a relevant example. However, unlike for Gaussian state-space models — where filtering, predictive and smoothing distributions are available in closed form — binary state-space models require approximations or sequential Monte Carlo strategies for inference and prediction. This is due to the apparent absence of conjugacy between the Gaussian states and the likelihood induced by the observation equation for the binary data. In this article we prove that the filtering, predictive and smoothing distributions in dynamic probit models with Gaussian state variables are, in fact, available and belong to a class of unified skew-normals (sun) whose parameters can be updated recursively in time via analytical expressions. Also the key functionals of these distributions are, in principle, available, but their calculation requires the evaluation of multivariate Gaussian cumulative distribution functions. Leveraging sun properties, we address this issue via novel Monte Carlo methods based on independent samples from the smoothing distribution, that can easily be adapted to the filtering and predictive case, thus improving state-of-the-art approximate and sequential Monte Carlo inference in small-to-moderate dimensional studies. Novel sequential Monte Carlo procedures that exploit the sun properties are also developed to deal with online inference in high dimensions. Performance gains over competitors are outlined in a financial application.

• Item Type: MPRA Paper
• Language: English
• Keywords: Dynamic probit model; Kalman filter; Particle filter; State-space model; sun
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C10 - General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C11 - Bayesian Analysis: General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C15 - Statistical Simulation Methods: General; C - Mathematical and Quantitative Methods > C4 - Econometric and Statistical Methods: Special Topics; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C53 - Forecasting and Prediction Methods ; Simulation Methods

References

Albert, J.H., Chib, S.: Bayesian analysis of binary and polychotomous response data. J. Am. Stat. Assoc. 88(422), 669–679 (1993)

Andrieu, C., Doucet, A.: Particle filtering for partially observed Gaussian state space models. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 64(4), 827–836 (2002)

Arellano-Valle, R.B., Azzalini, A.: On the unification of families of skew-normal distributions. Scand. J. Stat. 33(3), 561–574 (2006)

Arnold, B.C., Beaver, R.: Hidden truncation models. Sankhya Ser. A 62(1), 23–35 (2000)

Arnold, B.C., Beaver, R.J., Azzalini, A., Balakrishnan, N., Bhaumik, A., Dey, D., Cuadras, C., Sarabia, J.M.: Skewed multivariate models related to hidden truncation and/or selective reporting. Test 11(1), 7–54 (2002)

Atkins, A., Niranjan, M., Gerding, E.: Financial news predicts stock market volatility better than close price. J. Fin. Data Sci. 4(2), 120–137 (2018)

Azzalini, A., Bacchieri, A.: A prospective combination of phase II and phase III in drug development. Metron 68(3), 347–369 (2010)

Azzalini, A., Capitanio, A.: Statistical applications of the multivariate skew normal distribution. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 61(3), 579–602 (1999)

Azzalini, A., Capitanio, A.: The Skew-normal and Related Families. Cambridge University Press, Cambridge (2014)

Azzalini, A., Dalla Valle, A.: The multivariate skew-normal distribution. Biometrika 83(4), 715–726 (1996)

Botev, Z.: The normal law under linear restrictions: simulation and estimation via minimax tilting. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 79(1), 125–148 (2017)

Carlin, B.P., Polson, N.G., Stoffer, D.S.: A Monte Carlo approach to non-normal and nonlinear state-space modeling. J. Am. Stat. Assoc. 87(418), 493–500 (1992)

Chib, S., Greenberg, E.: Analysis of multivariate probit models. Biometrika 85(2), 347–361 (1998)

Chopin, N., Ridgway, J.: Leave Pima Indians alone: Binary regression as a benchmark for Bayesian computation. Stat. Sci. 32(1), 64–87 (2017)

Deligiannidis G., Doucet A., Rubenthaler S.: Ensemble rejection sampling (2020). arXiv preprint arXiv:2001.09188

Doucet, A., Johansen, A.M.: A tutorial on particle filtering and smoothing: fifteen years later. Handb. Nonlinear Filter. 12, 656–704 (2009)

Doucet, A., Godsill, S., Andrieu, C.: On sequential Monte Carlo sampling methods for Bayesian filtering. Stat. Comput. 10(3), 197–208 (2000)

Doucet, A., De Freitas, N., Gordon, N.: Sequential Monte Carlo Methods in Practice. Springer, Berlin (2001)

Durante, D.: Conjugate Bayes for probit regression via unified skew-normal distributions. Biometrika 106(4), 765–779 (2019)

Durbin, J., Koopman, S.J.: Time Series Analysis by State Space Methods. Oxford University Press, Oxford (2012)

Fasano A., Durante D.: A class of conjugate priors for multinomial probit models which includes the multivariate normal one (2021). arXiv preprint arXiv:2007.06944

Gelman, A., Jakulin, A., Pittau, M.G., Su, Y.S.: A weakly informative default prior distribution for logistic and other regression models. Ann. Appl. Stat. 2(4), 1360–1383 (2008)

González-Farías, G., Domínguez-Molina, A., Gupta, A.K.: Additive properties of skew normal random vectors. J. Stat. Plan. Infer. 126(2), 521–534 (2004)

Gordon, N.J., Salmond, D.J., Smith, A.F.: Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc-F. 140(2), 107–113 (1993)

Gupta, A.K., González-Farías, G., Domínguez-Molina, A.: A multivariate skew normal distribution. J. Multivariate Anal. 89(1), 181-190 (2004)

Gupta, A.K., Aziz, M.A., Ning, W.: On some properties of the unified skew-normal distribution. J. Stat. Theor. Pract. 7(3), 480–495 (2013)

Horrace, W.C.: Some results on the multivariate truncated normal distribution. J. Multivariate Anal. 94(1), 209–221 (2005)

Johndrow, J.E., Smith, A., Pillai, N., Dunson, D.B.: MCMC for imbalanced categorical data. J. Am. Stat. Assoc. 114(527), 1394-1403 (2019)

Julier S.J., Uhlmann J.K.: New extension of the Kalman filter to nonlinear systems. In: Proceedings SPIE 3068, Signal Processing, Sensor Fusion, and Target Recognition, pp. 182–194 (1997)

Kalman, R.E.: A new approach to linear filtering and prediction problems. J. Basic Eng. 82(1), 35–45 (1960)

Kara, Y., Boyacioglu, M.A., Baykan, Ö.K.: Predicting direction of stock price index movement using artificial neural networks and support vector machines: The sample of the Istanbul stock exchange. Expert Syst. Appl. 38(5), 5311–5319 (2011)

Keane, M.P., Wolpin, K.I.: Empirical applications of discrete choice dynamic programming models. Rev. Econ. Dynam. 12(1), 1–22 (2009)

Kim, K., Han, I.: Genetic algorithms approach to feature discretization in artificial neural networks for the prediction of stock price index. Expert Syst. Appl. 19(2), 125–132 (2000)

Kitagawa, G.: Monte Carlo filter & smoother for non-Gaussian nonlinear state space models. J. Comput. Gr. Stat. 5(1), 1–25 (1996)

Lin, M., Chen, R., Liu, J.S.: Lookahead strategies for sequential Monte Carlo. Stat. Sci. 28(1), 69–94 (2013)

Liu, J., Chen, R.: Sequential Monte Carlo methods for dynamic systems. J. Am. Stat. Assoc. 93(443), 1032–1044 (1998)

Liu, X., Daniels, M.J., Marcus, B.: Joint models for the association of longitudinal binary and continuous processes with application to a smoking cessation trial. J. Am. Stat. Assoc. 104(485), 429–438 (2009)

MacDonald, I.L., Zucchini, W.: Hidden Markov and Other Models for Discrete-Valued Time Series. CRC Press, Boca Raton (1997)

Pakman, A., Paninski, L.: Exact Hamiltonian Monte Carlo for truncated multivariate Gaussians. J. Comput. Gr. Stat. 23(2), 518–542 (2014)

Petris, G., Petrone, S., Campagnoli, P.: Dynamic Linear Models with R. Springer, Berlin (2009)

Pitt, M.K., Shephard, N.: Filtering via simulation: Auxiliary particle filters. J. Am. Stat. Assoc. 94(446), 590–599 (1999)

Shephard, N.: Partial non-Gaussian state space. Biometrika 81(1), 115–131 (1994)

Soyer, R., Sung, M.: Bayesian dynamic probit models for the analysis of longitudinal data. Comput. Stat. Data Anal. 68, 388–398 (2013)

Uhlmann, J.K.: Algorithms for multiple-target tracking. Am. Sci. 80(2), 128–141 (1992)

Villani, C.: Optimal Transport: Old and New. Springer Science & Business Media, Berlin (2008)

West, M., Harrison, J.: Bayesian Forecasting and Dynamic Models. Springer Science & Business Media, Berlin (2006)