Maximum Likelihood Estimation of the Multivariate Normal Mixture Model

Boldea, Otilia and Magnus, Jan R. (2009): Maximum Likelihood Estimation of the Multivariate Normal Mixture Model. Published in: Journal of the American Statistical Association , Vol. 104, No. 488 (2009): pp. 1539-1549.

Abstract

The Hessian of the multivariate normal mixture model is derived, and estimators of the information matrix are obtained, thus enabling consistent estimation of all parameters and their precisions. The usefulness of the new theory is illustrated with two examples and some simulation experiments. The newly proposed estimators appear to be superior to the existing ones.

• Item Type: MPRA Paper
• Language: English
• Keywords: Mixture model; Maximum likelihood; Information matrix
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C13 - Estimation: General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C10 - General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C15 - Statistical Simulation Methods: General

References

Aitken, M., and Rubin, D. B. (1985), “Estimation and Hypothesis Testing in Finite Mixture Models,” Journal of the Royal Statistical Society, Ser. B, 47, 67–75.

Ali, M. M., and Nadarajah, S. (2007), “Information Matrices for Normal and Laplace Mixtures,” Information Sciences, 177, 947–955.

Anderson, E. (1935), “The Irises of the Gaspe Peninsula,” Bulletin of the American Iris Society, 59, 2-5.

Basford, K. E., Greenway, D. R., McLachlan, G. J., and Peel, D. (1997), “Standard Errors of Fitted Means Under Normal Mixture Models,” Computational Statistics, 12, 1–17.

Behboodian, J. (1972), “Information Matrix for a Mixture of Two Normal Distributions,” Journal of Statistical Computation and Simulation, 1, 1–16.

Chesher, A. D. (1983), “The Information Matrix Test: Simplified Calculation via a Score Test Interpretation,” Economics Letters, 13, 15–48.

Davidson, R., and MacKinnon, J. G. (2004), Econometric Theory and Methods, New York: Oxford University Press. Day, N. E. (1969), “Estimating the Components of a Mixture of Normal Distributions,” Biometrika, 56, 463–474.

Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977), “Maximum Likelihood from Incomplete Data via the EM Algorithm” (with discussion), Journal of the Royal Statistical Society, Ser. B, 39, 1–38.

Dietz, E., and B¨ohning, D. (1996), “Statistical Inference Based on a General Model of Unobserved Heterogeneity,” in Advances in GLIM and Statistical Modeling, eds. L. Fahrmeir, F. Francis, R. Gilchrist, and G. Tutz, Lecture Notes in Statistics, Berlin: Springer, pp. 75-82.

Efron, B. (1979), “Bootstrap Methods: Another Look at the Jackknife,” The Annals of Statistics, 7, 1-26.

Efron, B., and Tibshirani, R. (1993), An Introduction to the Bootstrap, London: Chapman & Hall.

Fisher, R. A. (1936), “The Use of Multiple Measurements in Taxonomic Problems,” Annals of Eugenics, 7, 179–188.

Habbema, J. D. F., Hermans, J., and van den Broek, K. (1974), “A Step-Wise Discriminant Analysis Program Using Density Estimation,” in Proceedings in Computational Statistics, Compstat 1974, Wien: Physica Verlag, pp. 101–110.

Hathaway, R. J. (1985), “A Constrained Formulation of Maximum-Likelihood Estimation for Normal Mixture Distributions,” The Annals of Statistics, 13, 795–800.

Horowitz, J. L. (1994), “Bootstrap-Based Critical Values for the Information Matrix Test,” Journal of Econometrics, 61, 395–411.

Huber, P. J. (1967), “The Behavior of Maximum Likelihood Estimates under Non-Standard Conditions,” in Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1, eds. L. M. LeCam and J. Neyman, Berkeley: University of California Press, pp. 221–233.

Lancaster, A. (1984), “The Covariance Matrix of the Information Matrix Test,” Econometrica, 52, 1051–1053.

Liu, C. (1998), “Information Matrix Computation from Conditional Information via Normal Approximation,” Biometrika, 85, 973–979.

Louis, T. A. (1982), “Finding the Observed Information Matrix When Using the EM Algorithm,” Journal of the Royal Statistical Society, Ser. B, 44, 226–233.

Magnus, J. R. (1988), Linear Structures, Griffin’s Statistical Monographs and Courses, No. 42, London: Edward Arnold and New York: Oxford University Press.

Magnus, J. R., and Neudecker, H. (1988), Matrix Differential Calculus with Applications in Statistics and Econometrics, Chichester/New York: John Wiley, Second edition, 1999.

McLachlan, G. J., and Basford, K.E. (1988), Mixture Models: Inference and Applications to Clustering, New York: Marcel Dekker.

McLachlan, G. J., and Krishnan, T. (1997), The EM Algorithm and Extensions, New York: John Wiley.

McLachlan, G. J., and Peel, D. (2000), Finite Mixture Models, New York: John Wiley.

McLachlan, G. J., Peel, D., Basford, K. E., and Adams, P. (1999), “Fitting of Mixtures of Normal and t-Components,” Journal of Statistical Software, 4, Issue 2, www.maths.uq.edu.au/∼gjm/emmix/emmix.html.

Newton, M. A., and Raftery, A. E. (1994), “Approximate Bayesian Inference with the Weighted Likelihood Bootstrap” (with discussion), Journal of the Royal Statistical Society, Ser. B, 56, 3–48.

Newcomb, S. (1886), “A Generalized Theory of the Combination of Observations so as to Obtain the Best Result,” American Journal of Mathematics, 8, 343–366.

Pearson, K. (1894), “Contribution to the Mathematical Theory of Evolution,” Philosophical Transactions of the Royal Society, Ser. A, 185, 71–110.

Stigler, S. M. (1986), The History of Statistics: The Measurement of Uncertainty Before 1900, Cambridge, MA: Belknap.

Titterington, D. M., Smith, A. F. M., and Makov, U. E. (1985), Statistical Analysis of Finite Mixture Distributions, New York: John Wiley.

White, H. (1982), “Maximum Likelihood Estimation of Misspecified Models,” Econometrica, 50, 1–26.

Xu, L., and Jordan, M. I. (1996), “On Convergence Properties of the EM Algorithm for Gaussian Mixtures,” Neural Computation, 8, 129–151.