Latent Markov models: a review of a general framework for the analysis of longitudinal data with covariates

Bartolucci, Francesco and Farcomeni, Alessio and Pennoni, Fulvia (2012): Latent Markov models: a review of a general framework for the analysis of longitudinal data with covariates.

Abstract

We provide a comprehensive overview of latent Markov (LM) models for the analysis of longitudinal data. The main assumption behind these models is that the response variables are conditionally independent given a latent process which follows a first-order Markov chain. We first illustrate the more general version of the LM model which includes individual covariates. We then illustrate several constrained versions of the general LM model, which make the model more parsimonious and allow us to consider and test hypotheses of interest. These constraints may be put on the conditional distribution of the response variables given the latent process (measurement model) or on the distribution of the latent process (latent model). For the general version of the model we also illustrate in detail maximum likelihood estimation through the Expectation-Maximization algorithm, which may be efficiently implemented by recursions known in the hidden Markov literature. We discuss about the model identifiability and we outline methods for obtaining standard errors for the parameter estimates. We also illustrate methods for selecting the number of states and for path prediction. Finally, we illustrate Bayesian estimation method. Models and related inference are illustrated by the description of relevant socio-economic applications available in the literature.

• Item Type: MPRA Paper
• Language: English
• Keywords: EM algorithm, Bayesian framework, Forward-Backward recursions, Hidden Markov models, Measurement errors, Panel data, Unobserved heterogeneity
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C10 - General; C - Mathematical and Quantitative Methods > C3 - Multiple or Simultaneous Equation Models ; Multiple Variables > C33 - Panel Data Models ; Spatio-temporal Models

References

Akaike, H. (1973). Information theory as an extension of the maximum likelihood principle. In Petrov, B. N. and F., C., editors, Second International symposium on information theory, pages 267–281, Budapest. Akademiai Kiado.

Altman, R. M. (2007). Mixed hidden Markov models: an extension of the hidden Markov model to the longitudinal data setting. Journal of the American Statistical Association, 102:201–210.

Anderson, T. W. (1951). Probability models for analysing time changes in attitudes. In Corporation, T. R., editor, The use of mathematical models in the measurement of the attitudes.

Lazarsfelsd P. F. Anderson, T. W. (1954). Probability models for analysing time changes in attitudes. In F., L. P., editor, Mathematical Thinking in the Social Science. The Free press.

Andersson, S. and Rydén, T. (2009). Subspace estimation and prediction methods for hidden markov models. The Annals of Statistics, 37:4131–4152.

Archer, G. E. B. and Titterington, D. M. (2009). Parameter estimation for hidden markov chains. J. Statist. Plann. Inference, 108:365–390.

Bartolucci, F. (2006). Likelihood inference for a class of latent markov models under linear hypotheses on the transition probabilities. Journal of the Royal Statistical Society, series B, 68:155–178.

Bartolucci, F. and Farcomeni, A. (2009). A multivariate extension of the dynamic logit model for longitudinal data based on a latent markov heterogeneity structure. Journal of the American Statistical Association, 104:816–831.

Bartolucci, F., Lupparelli, M., and Montanari, G. E. (2009). Latent Markov model for binary longitudinal data: an application to the performance evaluation of nursing homes. Annals of Applied Statistics, 3:611–636.

Bartolucci, F. and Pennoni, F. (2007). A class of latent markov models for capture-recapture data allowing for time, heterogeneity and behavior effects. Biometrics, 63:568–578.

Bartolucci, F., Pennoni, F., and Francis, B. (2007). A latent markov model for detecting patterns of criminal activity. Journal of the Royal Statistical Society, Series A, 170:151–132.

Bartolucci, F., Pennoni, F., and Vittadini, G. (2011). Assessment of school performance through a multilevel latent Markov Rasch model. Journal of Educational and Behavioural Statistics, in press, 36:491–522.

Baum, L. and Petrie, T. (1966). Statistical inference for probabilistic functions of finite state Markov chains. Annals of Mathematical Statistics, 37:1554–1563.

Baum, L., Petrie, T., Soules, G., and Weiss, N. (1970). A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Annals of Mathematical Statistics, 41:164–171.

Berchtold, A. (2004). Optimization of mixture models: Comparison of different strategies. Computational statistics, 19:385–406.

Bernardo, J. M. and Smith, A. F. M. (1994). Bayesian Theory. Wiley, Chichester. Bickel, P. J., Ritov, Y., and Ryd ́en, T. (1998). Asymptotic normality of the maximum-likelihood estimator for general hidden Markov models. Annals of Statistics, 26:1614–1635.

Bollen, K. A. and Curran, P. J. (2006). Latent curve models: A structural equation perspective. Wiley, Hoboken, NJ.

Bornmann, L., Mutz, R., and Daniel, H.-D. (2008). Latent markov modeling applied to grand peer review. Journal of Informetrics, 2:217–228.

Boucheron, S. and Gassiat, E. (2007). An information-theoretic perspective on order estimation. In O. Cappé, E. Moulines, T. R., editor, Inference in Hidden Markov Models, pages 565–602. Springer.

Bye, B. V. and Schechter, E. S. (1986). A latent Markov model approach to the estimation of response error in multiwave panel data. Journal of the American Statistical Association, 81:375–380.

Capp ́e, O., Moulines, E., and Rydén, T. (1989). Inference in Hidden Markov Models. Springer-Verlag, New York.

Capp ́e, O., Moulines, E., and Rydén, T. (2005). Inference in Hidden Markov Models. Springer-Verlag New York, Inc., Secaucus, NJ, USA.

Cheng, R. C. H. and Liu, W. B. (2001). The consistency of estimators in finite mixture models. Scandinavian Journal of Statistics, 28:603–616.

Chib, S. (1996). Calculating posterior distributions and modal estimates in Markov mixture models. Journal of Econometrics, 75:79–97.

Collins, L. M. and Wugalter, S. E. (1992). Latent class models for stage-sequential dynamic latent variables. Multivariate Behavioral Research, 27:131–157.

Colombi, R. and Forcina, A. (2001). Marginal regression models for the analysis of positive association of ordinal response variables. Biometrika, 88:1007–1019.

Congdon, P. (2006). Bayesian model choice based on Monte Carlo estimates of posterior model probabilities. Computational Statistics & Data Analysis, 50:346–357.

Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977). Maximum likelihood from incomplete data via the em algorithm (with discussion). Journal of the Royal Statistical Society, Series B, 39:1–38.

Dias, J. G. and Vermunt, J. K. (2007). Latent class modeling of website users’ search patterns: Implications for online market segmentation. Journal of Retailing and Consumer Services, 14:359–368.

Elliot, D. S., Huizinga, D., and Menard, S. (1989). Multiple Problem Youth: Delinquency, Substance Use, and Mental Health Problems. Springer-Verlag, New York.

Feng, Z. and McCulloch, C. E. (1996). Using bootstrap likelihood ratios in finite mixture models. J. R. Statist. Soc., 58:609–617.

Fitzmaurice, G., Davidian, M., Verbeke, G., and G., M., editors (2009). Longitudinal data analysis. Chapman and Hall, CRC, London.

Fruhwirth-Schnatter, S. (2001). Markov chain Monte Carlo estimation of classical and dynamic switching and mixture models. Journal of the American Statistical Association, 96.

Ghahramani, Z. and Jordan, M. I. (1997). Factorial hidden markov models. Machine Learning, 29:245–273.

Glonek, G. F. V. and McCullagh, P. (1995). Multivariate logistic models. Journal of the Royal Statistical Society B, 57:533–546.

Goodman, L. A. (1961). Statistical methods for the mover-stayer model. Journal of the American Statistical Association, 56:841–868.

Goodman, L. A. (1974). Exploratory latent structure analysis using both identifiable and unidentifiable models. Biometrika, 61:215–231.

Green, P. J. (1995). Reversible jump Markov chain Monte Carlo computation and bayesian model determination. Biometrika, 82:711–732.

Hoffmann, L., Lehrke, M., and Todt, E. (1985). Development and changes in pupils’ interest in physics (grade 5 to 10): Design of a longitudinal study. In Lehrke, M., Hoffmann, L., and Gardner, P. L., editors, Interest in Science and Technology Education, pages 71–80, Kiel. IPN.

Juang, B. and Rabiner, L. (1991). Hidden Markov models for speech recognition. Technometrics, 33:251–272.

Kaplan, D. (2008). An overview of markov chain methods for the study of stage-sequential developmental processes. Developmental Psychology, 44:457–467.

Koski, T. (2001). Hidden Markov Models for Bioinformatics. Kluwer, Dordrecht.

Kunsch, H. R. (2005). State space and hidden markov models. In O. E. Barndorff-Nielsen, D. R. C. and C. Kluppelberg, e., editors, Complex Stochastic Systems, pages 109–173, Boca Raton, FL. Chapman and Hall/CRC.

Langeheine, R. (1988). New development in latent class theory. In Langeheine, R. and J., R., editors, Latent trait and latent class models, pages 77–108. New York: Plenum Press.

Langeheine, R. (1994). Latent variables markov models. In von Eye, A. and Clogg, C., editors, Latent variables analysis: Applications for developmental research, pages 373–395, Thousand Oaks, CA. Sage.

Langeheine, R. and Van de Pol, F. (1994). Discrete-time mixed markov latent class models. In Dale, A. and Davies, R., editors, Analyzing Social and Political Change: a Casebook of Methods, pages 171–197, London. Sage Publications.

Lazarsfeld, P. F. (1950). The logical and mathematical foundation of latent structure analysis. In S. A. Stouffer, L. Guttman, E. A. S., editor, Measurement and Prediction, New York. Princeton University Press.

Lazarsfeld, P. F. and Henry, N. W. (1968). Latent Structure Analysis. Houghton Mifflin, Boston. Leonard, T. (1975). Bayesian estimation methods for two-way contingency tables. Journal of the Royal Statistical Society (Ser. B), 37:23–37.

Levinson, S. E., Rabiner, L. R., and Sondhi, M. M. (1983). An introduction to the application of the theory of probabilistic functions of a Markov process to automatic speech recognition. Bell System Technical Journal, 62:1035–1074.

Louis, T. A. (1982). Finding the observed information matrix when using the EM algorithm. Journal of the Royal Statistical Society, Series B: Methodological, 44:226–233.

Lystig, T. C. and Hughes, J. (2002). Exact computation of the observed information matrix for hidden markov models. Journal of Computational and Graphical Statistics, 11:678–689.

MacDonald, I. L. and Zucchini, W. (1997). Hidden Markov and other Models for Discrete-Valued Time Series. Chapman and Hall, London.

Magidson, J. and Vermunt, J. K. (2001). Latent class factor and cluster models, bi-plots and related graphical displays. Sociological Methodology, 31:223–264.

Maruotti, A. (2011). Mixed hidden markov models for longitudinal data: An overview. International Statistical Review, 79.

McCullagh, P. and Nelder, J. A. (1989). Generalized Linear Models, 2nd Edition. Chapman and Hall, CRC, London.

McHugh, R. B. (1956). Efficient estimation and local identification in latent class analysis. Psy- chometrika, 21:331–347.

McLachlan, G. and Peel, D. (2000). Finite Mixture Models. Wiley.

Muthén, B. (2004). Latent variable analysis: Growth mixture modeling and related techniques for longitudinal data growth mixture modeling and related techniques for longitudinal data. In Kaplan, D., editor, Handbook of quantitative methodology for the social sciences, pages 345–368. Sage Publications, Newbury Park, CA.

Nagin, D. (1999). Analyzing developmental trajectories: A semi-parametric, group-based approach. Psychological Methods, 4:139–157.

Nazaret, W. (1987). Bayesian log-linear estimates for three-way contingency tables. Biometrika, 74:401–410.

Paas, L. J., Vermunt, J. K., and Bijlmolt, T. H. A. (2009). Discrete time, discrete state latent markov modelling for assessing and predicting household acquisitions of financial products. Jour- nal of the Royal Statistical Society, Series A, 170:955–974.

Rijmen, F., Vansteelandt, K., and De Boeck, P. (2007). Latent class models for diary methods data: parameter estimation by local computations. Psychometrika, 73:167–182.

Robert, C., Ryden, T., and Titterington, D. (2000). Bayesian inference in hidden Markov models through the reversible jump Markov chain Monte Carlo method. Journal of the Royal Statistical Society, Series B, 62:57–75.

Robert, C. P. and Casella, G. (2010). Monte Carlo Statistical Methods, 2nd Edition. Springer-Verlag, New York.

Roeder, K., Lynch, K. G., and Nagin, D. S. (1999). Modeling uncertainty in latent class membership: a case study in criminology. Journal of the American Statistical Association, 94:766–776.

Rost, J. (2002). Mixed and latent markov models as item response models. Methods of Psychological Research Online, Special Issue, pages 53–70.

Rusakov, D. and Geiger, D. (2005). Asymptotic model selection for naive Bayesian networks. Journal of Machine Learning Research, 6.

Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6:461–464.

Spezia, L. (2010). Bayesian analysis of multivariate gaussian hidden Markov models with an unknown number of regimes. Journal of Time Series Analysis, 31:1–11.

Turner, R. (2008). Direct maximization of the likelihood of a hidden Markov model. Computational Statistics and Data Analysis, 52:4147–4160.

Turner, T. R., Cameron, M. A., and Thomson, P. J. (1998). Hidden Markov chains in generalized linear models. The Canadian Journal of Statistics / La Revue Canadienne de Statistique, 26:107–125.

Tuyl, F., Gerlach, R., and Mengersen, K. (2009). Posterior predictive arguments in favor of the Bayes-Laplace prior as the consensus prior for binomial and multinomial parameters. Bayesian Analysis, 4:151–158.

van de Pol, F. and Langeheine, R. (1990). Mixed markov latent class models. Sociological Methodology, 20:213–247.

Vansteelandt, K., Rijmen, F., Pieters, G., and Vanderlinden, J. (2007). Drive for thinness, affect regulation and physical activity in eating disorders: a daily life study. Behaviour Research and Therapy, 45:1717–1734.

Vermunt, J. (2010). Longitudinal research with latent variables. In van Montfort, K., Oud, J., and Satorra, A., editors, Handbook of Advanced Multilevel Analysis, pages 119–152. Springer, Heidelberg, Germany.

Vermunt, J. K., Langeheine, R., and B ̈ockenholt, U. (1999). Discrete-time discrete-state latent markov models with time-constant and time-varying covariates. Journal of Educational and Behavioral Statistics, 24:179–207.

Viterbi, A. (1967). Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Transactions on Information Theory, 13:260–269.

Welch, L. R. (2003). Hidden Markov models and the Baum-Welch algorithm. IEEE Information Theory Society Newsletter, 53:1–13.

Wiggins, L. (1973). Panel Analysis: Latent probability models for attitude and behaviours processes. Elsevier, Amsterdam.

Wiggins, L. M. (1955). Mathematical models for the Analysis of Multi-wave Panels. PhD thesis, Columbia University, Ann Arbor.

Zucchini, W. and MacDonald, I. L. (2009). Hidden Markov Models for time series: an introduction using R. Springer-Verlag, New York.